<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Minkyu Choi on Medium]]></title>
        <description><![CDATA[Stories by Minkyu Choi on Medium]]></description>
        <link>https://medium.com/@minkyu.choi?source=rss-4051f131b5bd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*urx0oiUKAu5uAt6xOlXbag.png</url>
            <title>Stories by Minkyu Choi on Medium</title>
            <link>https://medium.com/@minkyu.choi?source=rss-4051f131b5bd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 19:42:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@minkyu.choi/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Reparameterization Trick]]></title>
            <link>https://medium.com/@minkyu.choi/reparameterization-trick-694dc5ed62db?source=rss-4051f131b5bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/694dc5ed62db</guid>
            <category><![CDATA[reparameterization-trick]]></category>
            <category><![CDATA[vae]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Minkyu Choi]]></dc:creator>
            <pubDate>Fri, 15 Sep 2023 18:36:18 GMT</pubDate>
            <atom:updated>2023-09-15T18:37:30.484Z</atom:updated>
            <content:encoded><![CDATA[<p>The reparameterization trick has indeed been introduced to allow gradient-based optimization (i.e., making it trainable) of models involving stochastic variables by transforming the sampling process into a differentiable operation.</p><p>But why we can’t backpropagate the gradient from models involving stochastic variable?</p><p>Sampling, at its core, introduces a source of randomness. When you sample from a distribution, you’re introducing a non-deterministc step. In terms of neural networks and optimization, gradient-based optimization relies on being able to compute the gradient of the loss with respect to the parameters. This requires the operations in the computation to be differentiable.</p><p>When we introduce a non-deterministic operation, like sampling, the exact relationship between the parameters of the distribution (like the mean and variance in a Gaussian) and the outcomes of the sampling operation is not well-defined. This makes it hard to compute a gradient. Simply put, the gradient of a truly random process is undefined because the process is not a function of its parameters in a conventional sense.</p><p>This is where the reparameterization trick becomes valuable. It allows us to reframe the problem in a way that “bypasses” the non-differentiable sampling step. Instead of directly sampling from the distribution, we sample from a standard distribution (like a standard Gaussian) and then deterministically transform that sample in a way that’s equivalent to sampling from the desired distribution. This transformation is differentiable with respect to the parameters of the target distribution, which allows for gradient-based optimization.</p><p>The reparameterization trick is a technique prmarily used in variational auto encoders (VAEs) to enable back propagation through random nodes as explanied above. It sample epsilon ε from 𝐍(0,I) instead of directly sampling from 𝓏 from 𝐍(μ,Σ), where μ is mean and Σ is covariance matrix.</p><p>The equation is 𝓏 = μ+ Lε, where L is the Cholesky decomposition of Σ, such that Σ = LL^T.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/372/1*k1k6e0ghkY9rZVnbNrsYuA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*zOWWdSZNqfvJxNvE4rdLYg.png" /><figcaption>Source: Slike 12 in Kingma’s NIPS 2015 workshop talk</figcaption></figure><p>This technique can work with other probabilistic distribution, not only Gaussian.</p><p>If you want to know more about autor: <a href="https://minkyuchoi-07.github.io/about/">Minkyu’s Blo</a>g</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=694dc5ed62db" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Distill Reinforcement Learning]]></title>
            <link>https://medium.com/@minkyu.choi/distill-reinforcement-learning-a3c6683f6919?source=rss-4051f131b5bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/a3c6683f6919</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Minkyu Choi]]></dc:creator>
            <pubDate>Tue, 04 Apr 2023 04:23:34 GMT</pubDate>
            <atom:updated>2023-04-04T04:23:34.807Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Reinforcement Learning</h3><p>Consider an agent situated in an unfamiliar setting, capable of garnering rewards through engagment with its surroundings. The agent’s objective is to perform actions that optimize the accumulation of rewards. Practical examples of this concept incldue a gaming bot striving for top scors or a robot executing physical tasks with tanglible objects, although the possibilities extend beyond these instances.</p><p>The primary objective of Reinforcement Learning (RL) is to develop an effective strategy for an agent through iterative trials and straightforward feedback. By emplying the optimial strategy, the agent can dynamically adpat to the environment, thereby maximizing future rewards.</p><p>Let’s delve into some fundamental concepts in RL.</p><p>An agent operates within an environment, and the way the environment, and the way the environment reacts to specific actions is dictacted by a model, which may be known or unknown to us. The agent can occupy on of many state (s∈S) within the environment and choose to take one of several actions (a∈A) to transition between states. The transition probabilities between state (P) determine the agents’ destination state. Upon taking an action, the environment offers a reward (r∈R) as feedback.</p><p>The model outlines the reward function and transition probabilities. Depending on whether we know the model’s working or not, there are two distnct situations.</p><p><strong>Known Model:</strong> Execute model-based RL with perfect inffomration for planning. When the environment is fully understood, the optimal solution can be found using Dynamic Programming (DP). This is reminiscent of solving problems such as the “longest increasing subsequence” or “traveling salesman problem.”</p><p><strong>Unknown Model:</strong> Perform model-free RL or attempt to explicitly learn the model as part of the algorithm, given that information is incomplete. The majority of the subsequent content addresses scenarios where the model is unknown.</p><p>The agent’s policy, π(s), offeres guidance on the optimal action to take in a specific state to maximize total rewards. Each state is linked to a value function, V(s), which forecasts the anticipated amount of future rewards obtainable in that state by following the corresponding policy. In essence, the value funcution measures a state’s quality. Both policy and value functions are the targets of reinforcement learning.</p><p>The interaction between the agent and the environment involves a series of actions and observed reward over time, t=1,2,…,T. Throughout this process, the agent accumulates knowledge about the environment, learns the optimial policy, and decides on the next action to take to efficiently learn the best policy. Let’s denote the state, action, and reward at time step t as St,At and Rtrespectively. Therefore, the interaction sequence is fully represented by a single <strong>episode</strong> (also referred to as “trial” or “trajectory”) that concludes at the terminal state ST: S1,A1,R1,S2,A2,R2,…,ST</p><p>If you want to know more:</p><p><a href="https://minkyuchoi-07.github.io/2023/04/02/distill-rl/">More</a>: <a href="https://minkyuchoi-07.github.io/2023/04/02/distill-rl/">https://minkyuchoi-07.github.io/2023/04/02/distill-rl/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a3c6683f6919" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning DevOps]]></title>
            <link>https://medium.com/@minkyu.choi/machine-learning-devops-e71794548bda?source=rss-4051f131b5bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/e71794548bda</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Minkyu Choi]]></dc:creator>
            <pubDate>Wed, 01 Apr 2020 19:30:21 GMT</pubDate>
            <atom:updated>2020-04-01T19:33:20.871Z</atom:updated>
            <content:encoded><![CDATA[<p>We’ve had a good DevOps definition for regular software development since there has been many good tools available in the market such as GitLab Jenkins and many other CI/CD automation tools. This is good for web or mobile applications which follow a traditional way to develop and deploy. However, many modern software now take in place to contain machine learning (ML) models to be more capable, if software have ML capability, we call it Artificial Intelligence aka AI/ML based application.</p><p>An AI/ML based application has two major components, a software application which can host a ML model and a ML model itself. A ML model is a major part of a software application. Without ML model, a software application is just a regular application.</p><p>Therefore, it is important to support a DevOps cycle for both software and a ML model. <strong>In AI/ML DevOps, these two developments cycle has to be integrated.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*qt-PhmNQBhcr0Z-NMbV24A.png" /></figure><p>In order to have a full DevOps for Artificial Intelligence Application, we need several different pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*glj8WokS6G8-6mJt0cjB5A.png" /></figure><ol><li>Data Pipeline: Data pipeline allow us to bring data from external sources; we could set up required ETL steps in Data Pipeline — assuming that we already know how to condition/process data.</li><li>Machine Learning Pipeline: This is where we could experiment and develop ML models, and also at the end, we could automate ML development cycle with Auto ML Tools such as Kubeflow</li><li>Software Pipeline: This is a traditional CI/CD pipeline where we could develop and deploy software application.</li><li>Integrated Pipeline (AI/ML DevOps): We need to integrate Machine Learning Pipeline into Software Pipeline so that every step can be integrated in one pipeline.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uuucefa1q2SoVLSSC2huxA.png" /></figure><p><strong>Step1</strong>: We need to set up a data pipeline first. It doesn’t matter where data is currently located. And if we don’t have a ETL specification — that is fine. We can set up full data pipeline later. However, we should at least have data ingest pipeline set up into our system so that we could access data from our environment.</p><p>Once we get data, we could do Exploratory Data Analysis (<strong>EDA</strong>) &amp; Feature Engineering. Afterward, we could fully automate step 1 process by setting ETL pipeline.</p><p><em>A. Ingest Data<br>B. Condition/Process Data (ETL)<br>C. Feature Engineering (ETL)<br>D. Loading Training Data (ETL)</em></p><p><strong>Step 2: </strong>This is where ML model experiment and development is happening. Users can experiment various models and develop the model on training dataset. If ML model is successfully developed, we could test and evaluate the models against validating and testing dataset and store the final model to model repository.</p><p><strong>Step 3: </strong>If we don’t need to automate this machine learning process we could skip step 3. However, if we somehow need to update and retrain the ML model regular bases then we would like to set to Auto ML pipeline. There are many different ways to set up the Auto ML pipeline. For example, Azure has it’s native service for this capability; AWS also has a similar capability something called SageMaker In my example, I am assuming that we are using Kubeflow . In order to use Kubeflow each ML step has to be containerized, and each container will be fitted into the pipeline. Finally Kubeflow will orchestrate all steps of ML development from data ingestion to model deploy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B-tQ8HYqjSCC6fxvc0fCzg.png" /></figure><p><strong>Step 4</strong>: It doesn’t need to be step 4. It’s just a separate pipeline. It can be worked Parallel with ML Pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Eqgeg0_6s4hno0MqgeTeQ.png" /></figure><p><strong>Step 5</strong>: If every pipeline has been matured meaning we know and understand clearly what we are doing, then you could integrate everything into one pipeline. We basically need to hook up model repo to our software pipeline and need to add little bit additional testing stages on our CI/CD Pipeline so that we could make sure that our model is incorporated well with software.</p><p>If we put everything together it will look like this:</p><p>I understand that it doesn’t look like 100% fully automated pipeline. This is because there are many different dependencies involved in Artificial Intelligence Application such as data, ML model and software. However, this DevOps pipeline will allow users not to spend so much time to re-train and re-use the ML development cycle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bnkMXpE7BVo0FrfUdGpBrw.png" /></figure><p><strong>When it comes to retrain or change the model, every step has to be re-configured and processed manually starting from a beginning.</strong></p><p>However, we could reduce the amount of effort by following the AI/ML DevOps cycle because Auto ML Pipeline will enable users to re-use their ML development cycle. If new data comes in, then users can simply trigger a pipeline — it also can be automated. Also, if there are issues on a cycle, users need to make a change of elements of a pipeline, not the entire processes, and re-trigger a pipeline. And the new model will be easily integrated on <strong>AI/ML DevOps pipeline.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e71794548bda" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Naive Bayes Classifier from scratch]]></title>
            <link>https://medium.com/@minkyu.choi/naive-bayes-classifier-from-scratch-e0f107bd2964?source=rss-4051f131b5bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/e0f107bd2964</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[machine-leraning]]></category>
            <category><![CDATA[naive-bayes]]></category>
            <category><![CDATA[classification]]></category>
            <dc:creator><![CDATA[Minkyu Choi]]></dc:creator>
            <pubDate>Tue, 24 Mar 2020 01:38:41 GMT</pubDate>
            <atom:updated>2023-04-09T04:57:09.169Z</atom:updated>
            <content:encoded><![CDATA[<h3>Use case - Spam filter</h3><p>We will use the Naive Bayes algorithm to fit a spam filter.</p><p>Spam filters are used in email services to classify received emails as &quot;Spam&quot; or &quot;Not Spam&quot;. A simple approach involves maintaining a vocabulary of words that commonly occur in &quot;Spam&quot; emails and classifying an email as &quot;Spam&quot; if the number of words from the dictionary that are present in the email is over a certain threshold.</p><p>Assume we are given the vocabulary consists of 15 words</p><p>V​ = {secret, offer, low, price, valued, customer, today, dollar, million, sports, is, for, play, healthy, pizza}</p><p>We will use ​ to represent the ith word in ​. As our training dataset, we are also given 3 example spam messages:</p><p>• million dollar offer</p><p>• secret offer today</p><p>• secret is secret</p><p>and 4 example non-spam messages</p><ul><li>low price for valued customer</li><li>play secret sports today</li><li>sports is healthy</li><li>low price pizza</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1479acc51b08f7f91b91dd9f6fc6228d/href">https://medium.com/media/1479acc51b08f7f91b91dd9f6fc6228d/href</a></iframe><p>train is our input vector x corresponding to each training message, and it has length n = 15 (length of V). Since we have 7 training example of message, We will have 7 by 15 training data - 7 data 15 features.</p><pre>train = np.zeros((7, 15))<br>label = [0,0,0,1,1,1,1]</pre><p>Converting 7 training message data set to x vector which has length n = 15</p><pre>for i in range(len(train)):<br>    for j in range(len(V)):<br>        for k in range(len(message[i].split(&quot; &quot;))):<br>            if V[j] == message[i].split(&quot; &quot;)[k]:<br>                train[i,j] += 1</pre><p>This is the feature x matrix.<br> • row = n message<br> • col = i_th feature vector</p><pre>([[0., 1., 0., 0., 0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0.],<br> [1., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],<br> [2., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.],<br> [0., 0., 1., 1., 1., 1., 0., 0., 0., 0., 0., 1., 0., 0., 0.],<br> [1., 0., 0., 0., 0., 0., 1., 0., 0., 1., 0., 0., 1., 0., 0.],<br> [0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 0., 0., 1., 0.],<br> [0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.]])</pre><p>Let’s list them separately</p><pre>for i in range(len(label)):<br>    if label[i] == 0:<br>        print(&quot;this is {}th message - It&#39;s SCAM&quot;.format(i))<br>        print(train[i])<br>    else:<br>        print(&quot;this is {}th message - It&#39;s NOT SCAM&quot;.format(i))<br>        print(train[i])</pre><pre>this is 0th message - It&#39;s SCAM<br>[0. 1. 0. 0. 0. 0. 0. 1. 1. 0. 0. 0. 0. 0. 0.]<br>this is 1th message - It&#39;s SCAM<br>[1. 1. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]<br>this is 2th message - It&#39;s SCAM<br>[2. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]<br>this is 3th message - It&#39;s NOT SCAM<br>[0. 0. 1. 1. 1. 1. 0. 0. 0. 0. 0. 1. 0. 0. 0.]<br>this is 4th message - It&#39;s NOT SCAM<br>[1. 0. 0. 0. 0. 0. 1. 0. 0. 1. 0. 0. 1. 0. 0.]<br>this is 5th message - It&#39;s NOT SCAM<br>[0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 1. 0. 0. 1. 0.]<br>this is 6th message - It&#39;s NOT SCAM<br>[0. 0. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]</pre><p><strong>Training Bayes Classifier</strong></p><p>spam and ham array will be count of x_i when each message is spam or not spam.<br> • spam = (Y=0|x_i)<br> • ham = (Y=1|x_i)</p><pre>spam = np.zeros((1, 15))<br>ham = np.zeros((1, 15))</pre><pre>for c in range(len(label)):<br>    for i in range(train.shape[1]):<br>        if label[c] == 0: # Spam<br>            spam[0,i] += train[c,i]<br>            <br>        else: # Not Spam<br>            ham[0,i] += train[c,i]</pre><p>OUTPUT:</p><pre>([[3., 2., 0., 0., 0., 0., 1., 1., 1., 0., 1., 0., 0., 0., 0.]])</pre><p>If you see a result above, spam[0] = 3 ,spam[1] = 2 , spam [2] = 0, it means total count of x0 = secret is three among training data classified as <em>spam</em>. We have two offer and zeor low in our training data.</p><p>In this section, we will calculate probabiliy of each word xi when a message is spam or not.<br> • prob_spam = P(x_i|y=0)<br> • prob_ham = P(x_i|y=1)</p><pre>spam_counter = 0<br>ham_counter = 0</pre><pre>for c in range(len(label)):<br>    if label[c] == 0:<br>        spam_counter += 1<br>        <br>    else:<br>        ham_counter += 1<br>        <br>prob_spam_x_i = spam/spam_counter<br>prob_ham_x_i = ham/ham_counter</pre><p>OUTPUT:</p><pre>prob_spam_x_i<br>array([[1.00, 0.66, 0.00, 0.00, 0.00, 0.00, 0.33, 0.33, 0.33, 0.00        ,0.33, 0.00, 0.00, 0.00, 0.00]])</pre><pre>prob_ham_x_i</pre><pre>array([[0.25, 0.  , 0.5 , 0.5 , 0.25, 0.25, 0.25, 0.  , 0.  , 0.5 , 0.25,0.25, 0.25, 0.25, 0.25]])</pre><p><strong>Bayes Theorem</strong><br>Calculating P(y=0|x_i)<br>prob_spam_per_word = P(y=0|x_i)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/50b75434ea0abf3c2530c9640c6c800b/href">https://medium.com/media/50b75434ea0abf3c2530c9640c6c800b/href</a></iframe><p>OUTPUT:</p><pre>prob_spam_per_word<br>array([[0.75, 1.00, 0.00, 0.00, 0.00, 0.00, 0.50, 1.00, 1.00, 0.00, 0.50,0.00, 0.00, 0.00, 0.00]])</pre><p>The result above means that the probability of spam per i_th words. For example, the probability of spam when we have secret is 75% and a probability of spam of offer is 100%.</p><pre>prob_ham_per_word # This is the probability of ham per words.</pre><pre>array([[0.25, 0.  , 1.  , 1.  , 1.  , 1.  , 0.5 , 0.  , 0.  , 1.  , 0.5 , 1.  , 1.  , 1.  , 1.  ]])</pre><p><strong>Since we don&#39;t have enough data, I will just use training dataset to evaluate the Bayes classifier</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/058266d1f544e1955df3767c4bbd49fb/href">https://medium.com/media/058266d1f544e1955df3767c4bbd49fb/href</a></iframe><pre>OUTPUT:</pre><pre>0 th email is SPAM with a probability of being spam 100.0%<br>1 th email is SPAM with a probability of being spam 100.0%<br>2 th email is NOT SPAM with a probability of being spam 42.85%<br>3 th email is NOT SPAM with a probability of being spam 0.0%<br>4 th email is NOT SPAM with a probability of being spam 0.0%<br>5 th email is NOT SPAM with a probability of being spam 0.0%<br>6 th email is NOT SPAM with a probability of being spam 0.0%</pre><h4><strong>Predicting</strong></h4><p>Given a new message “today is secret”, decide whether it is spam or not spam, based on the Naive Bayes classifier, learned from the above data.</p><pre>&#39;&#39;&#39;<br>&quot;today is secret&quot;<br>Vectorizing the email.. <br>&#39;&#39;&#39;</pre><pre>target = np.array([1., 0., 0., 0., 0., 0., 1., 0., 0., 0., 1., 0., 0., 0., 0.])</pre><p>Please play around with different threshold.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2da4677c3cb156141e07abad66bbb6de/href">https://medium.com/media/2da4677c3cb156141e07abad66bbb6de/href</a></iframe><p>OUTPUT:</p><pre>6 th email is SPAM with a probability of being spam 69.23%</pre><p><a href="https://minkyuchoi-07.github.io/2020/03/23/Naive%20Bayes%20from%20scratch/">Blog</a>: <a href="https://minkyuchoi-07.github.io/2020/03/23/Naive%20Bayes%20from%20scratch/">https://minkyuchoi-07.github.io/2020/03/23/Naive%20Bayes%20from%20scratch/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e0f107bd2964" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Feature Engineering 101]]></title>
            <link>https://medium.com/@minkyu.choi/test-f7d33c2b16bc?source=rss-4051f131b5bd------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7d33c2b16bc</guid>
            <category><![CDATA[feature-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Minkyu Choi]]></dc:creator>
            <pubDate>Mon, 23 Mar 2020 20:49:15 GMT</pubDate>
            <atom:updated>2023-04-09T04:56:19.380Z</atom:updated>
            <content:encoded><![CDATA[<p>If you ask yourself what’s the most important thing in machine learning, what’s your answer? All data scientist would have different answers.</p><p>Among the other many answers, I believe feature engineering is one of the most important in machine learning. Sometimes, it’s a more critical step than a model selection and training a model because a model cannot improve a model itself even though we put a lot of effort on a hyper parameter turning. However, well selected/extracted features could be applied to many different models and improve a performance.</p><p>A feature engineering includes feature selection and feature extraction. A feature selection is trial-error process to select relevant features from existing features. Since all features are simply selected from original features it’s easy to interpret what those features means. However, it is difficult to consider a relationship in selected features.</p><p>On the other hands, a feature extraction is more like functional process to extract relevant features from existing features. It requires a form of function which enable an algorithm to create/extract a new set of features. A relationship between features will be considered and number of features could be significantly reduced. Yet, an interpretation of extracted features is not easy.</p><p>We should use different methods of feature engineering depending on a machine learning algorithm we want to use.</p><p>In supervised learning, we could select features form Information gain, Stepwise regression, LASSO, Genetic algorithms, etc. If we want to extract features, Partial Least Squares (PLS) is an option.</p><p>In unsupervised learning, we could do a feature selection with PCA loading; a feature extraction uses Principal component analysis (PCA), Wavelets transforms, Autoencoder, etc.</p><p><a href="https://minkyuchoi-07.github.io/2020/01/23/feature-engineering-101/">Blog</a>: <a href="https://minkyuchoi-07.github.io/2020/01/23/feature-engineering-101/">https://minkyuchoi-07.github.io/2020/01/23/feature-engineering-101/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7d33c2b16bc" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>