<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Melanee Group on Medium]]></title>
        <description><![CDATA[Stories by Melanee Group on Medium]]></description>
        <link>https://medium.com/@melanee-melanee?source=rss-8273fce06f66------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*d0Rgdbb78GSYBhvfaRPCjQ.png</url>
            <title>Stories by Melanee Group on Medium</title>
            <link>https://medium.com/@melanee-melanee?source=rss-8273fce06f66------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 02:22:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@melanee-melanee/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The “electronic Babylonian Library” (eBL), Gilgamesh project]]></title>
            <link>https://levelup.gitconnected.com/the-electronic-babylonian-library-ebl-gilgamesh-project-f883e0ff068f?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/f883e0ff068f</guid>
            <category><![CDATA[history]]></category>
            <category><![CDATA[gilgamesh]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[archeaology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Thu, 23 May 2024 21:19:00 GMT</pubDate>
            <atom:updated>2024-07-20T11:41:20.673Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="electronic Babylonian Library" src="https://cdn-images-1.medium.com/max/1024/1*ouKwwWBSw1hO87476S6k8g.png" /><figcaption>Photo taken from <a href="https://github.com/ElectronicBabylonianLiterature/cuneiform-ocr">eBL OCR</a> project</figcaption></figure><p>Prof<a href="https://www.assyriologie.uni-muenchen.de/personen/professoren/jimenez/index.html"> <strong>Enrique Jiménez</strong></a> and his team at Ludwig Maximilian University (LMU) are utilizing advanced <strong>artificial intelligence </strong>(AI) and computer vision techniques to analyze and decipher ancient cuneiform inscriptions, including the Epic of <strong>Gilgamesh</strong>. They have developed several tools and algorithms to facilitate this process, which are available in their <a href="https://github.com/ElectronicBabylonianLiterature">GitHub</a> repositories.</p><p>He said:</p><p>“It’s a tool that didn’t exist before, a huge database of fragments. We believe it can play a vital role in reconstructing Babylonian literature, allowing us to make much faster progress.” Aptly named the <strong>Fragmentarium</strong>, it is designed to piece together fragments of text using systematic, automated methods. The designers expect that the program will also be able to identify and transcribe photos of cuneiform scripts in the future. To date, thousands of additional cuneiform fragments have been photographed in collaboration with the <a href="https://www.britishmuseum.org/collection"><strong>British Museum</strong></a> in London and the <strong>Iraq Museum</strong> in Baghdad. [<a href="https://www.lmu.de/en/newsroom/news-overview/news/playing-with-the-source-of-world-literature.html">Ref</a>]</p><p>Prof Enrique Jimenez said:</p><blockquote>“Everybody will be able to play around with the Fragmentarium. There are thousands of fragments that have not yet been identified”</blockquote><p>This project leverages Optical Character Recognition (OCR) and Natural Language Processing (NLP) technologies to read and match the texts. Specifically, they use OCR to convert the cuneiform signs from images into machine-readable text. Prof Enrique Jiménez’s team then apply algorithms to detect and match overlapping segments of different manuscripts, aiding in the reconstruction of fragmented texts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*eV6c8uPkJyt46eyH2HhDPg.png" /><figcaption>Tablets from Ashurbanipal’s Library, British museum [<a href="https://www.britishmuseum.org/blog/library-fit-king#:~:text=The%20&#39;Library%20of%20Ashurbanipal&#39;%20is,in%20Mesopotamia%20(ancient%20Iraq).">Ref</a>]</figcaption></figure><h3>How Prof Enrique Jimenez reconstructed Babylonian fragments?</h3><p>In his speech in LMU university, he said [<a href="https://www.youtube.com/watch?v=4QS9oCNUcRY">Ref</a>]:</p><p>“We have only a few pieces of each of these manuscripts. In this way, the Mesopotamia literature can be reconstructed and this is how we formulated the problem here at LMU. In this example, we have a fragment of a manuscript that we call manuscript A 1. If we are lucky we will find a fragment of another manuscript of the same text, which partially <strong>overlaps</strong> with A 1 and we will call it manuscript B. So there are two copies of the same text, so two independent manuscripts. This identification will enable us to identify another fragment of our first manuscript, manuscript A 1. The new fragments can be used to identify other parts of manuscript B, and a from that, so the next will grow. “</p><figure><img alt="Electronic Babylonian literature" src="https://cdn-images-1.medium.com/max/732/1*40TlFKnVXX18_vHWOLJMsw.png" /><figcaption>Overlapping manuscripts, photo taken from Prof Enrique YouTube video [<a href="https://www.youtube.com/watch?v=4QS9oCNUcRY">Ref</a>]</figcaption></figure><p>He continued:</p><p>“Solving puzzles is a hot topic for <a href="https://en.wikipedia.org/wiki/Computer_vision"><strong>computer vision</strong></a>. In this work, you create an algorithm for the reconstruction of broken archaeological finds.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OMpb1xOp4UqeMQeYF66AEw.jpeg" /><figcaption>an example of text puzzle [<a href="https://www.litographs.com/pages/puzzles-press">Ref</a>]</figcaption></figure><p>The <a href="https://github.com/ElectronicBabylonianLiterature">GitHub</a> organization of this project includes various tools and datasets, the most important ones are:</p><p><strong>ebl-api</strong>: The API for accessing the Electronic Babylonian Literature database.</p><p><strong>cuneiform-ocr</strong>: Tools for performing OCR on cuneiform texts.</p><p><strong>ngram-matcher</strong>: Algorithms for matching n-grams to identify overlapping text segments.</p><p>These tools collectively enable the identification and matching of text fragments by comparing pixel patterns and character sequences, allowing researchers to piece together ancient manuscripts from scattered fragments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yn-a2ybQYctxXOlrR7qebQ.png" /><figcaption>ngram-mathcher repository in <a href="https://github.com/ElectronicBabylonianLiterature/ngram-matcher">Github</a></figcaption></figure><p>The ngram-matcher repository in the <a href="https://www.ebl.lmu.de/fragmentarium">Electronic Babylonian Literature</a> (eBL) project is designed to identify overlapping text segments by matching n-grams, which are contiguous sequences of n items from a given sample of text or speech. This approach is useful for comparing fragments of ancient texts to find overlaps and reconstruct larger portions of manuscripts.</p><p>Key Concepts and Algorithms:</p><p>1. N-grams: These are sequences of ’n’ characters or words used for text comparison. For example, in the text “Gilgamesh”, the 2-grams (bigrams) would be “Gi”, “il”, “lg”, “ga”, “am”, “me”, “es”, “sh”.</p><p>2. Text Tokenization: The text is tokenized into n-grams. This means breaking down the text into smaller pieces that can be compared against other pieces.</p><p>3. String Matching Algorithms: Algorithms like Levenshtein distance, Jaccard similarity, or cosine similarity are used to compare the n-grams from different text fragments to identify overlaps. These algorithms quantify the similarity between two sequences by calculating the minimum number of single-character edits (insertions, deletions, or substitutions) required to change one sequence into the other.</p><p>4. Data Structures: Efficient data structures such as suffix trees or tries are used to store and quickly query the n-grams.</p><p>5. Scoring and Ranking: Matches are scored based on their similarity, and the highest scoring matches are considered the best candidates for overlapping text.</p><p><strong>Implementation in eBL:</strong></p><p>In the eBL project, the ngram-matcher is integrated with a larger framework that includes:</p><ul><li>Digitized Text Repositories: Large collections of transliterated texts from ancient manuscripts.</li><li>Pre-processing Pipelines: Tools for cleaning and preparing the text data, including removing noise and normalizing the text.</li><li>Matching Engine: The core n-gram matching algorithms are implemented here, which systematically compare all possible pairs of text fragments.</li><li>Post-processing: Results are filtered and verified, and matches are cross-referenced with existing scholarly work to ensure accuracy.</li></ul><p>At the end of this article let’s translate one of Gilgamesh fragments [<a href="https://www.ebl.lmu.de/corpus/L/1/4/SB/I">Ref</a>]:</p><figure><img alt="Electronic Babylonian Literature Gilgamesh inscription" src="https://cdn-images-1.medium.com/max/1024/1*eaknQwKSrwV2XCgP_7g1bg.jpeg" /><figcaption>One of Gilgamesh inscriptions [<a href="https://www.ebl.lmu.de/fragmentarium/K.2756.C?tab=photo">Ref</a>]</figcaption></figure><h4>The first 10 lines to English:</h4><p>1. [He who saw the Deep, the] foundation of the country,</p><p>2. [who knew all ways], was wise in everything!</p><p>3. [Gilgameš, who] saw the Deep, the foundation of the country,</p><p>4. [who] knew [all ways], was wise in everything!</p><p>5. [He explored] everywhere the seats [of power,]</p><p>6. he [knew] the totality of wisdom about everything.</p><p>7. He saw the secret and uncovered the hidden,</p><p>8. he brought back a message from the antediluvian age.</p><p>9. He came a distant road and was weary but granted rest,</p><p>10. all (his) labours [were] set down on a tablet of stone.</p><h4>Translate to Persian:</h4><ol><li>[او که ژرفا را دید،] بنیاد کشور،</li><li>[که همه راه‌ها را می‌دانست،] در همه چیز خردمند بود!</li><li>[گیلگمش، که] ژرفا را دید، بنیاد کشور،</li><li>[که] همه [راه‌ها را] می‌دانست، در همه چیز خردمند بود!</li><li>[او همه جا را] برای نشستن [قدرت‌ها] بررسی کرد،</li><li>او [مجموعه کامل] حکمت درباره همه چیز را می‌دانست.</li><li>او راز را دید و پنهان را آشکار کرد،</li><li>او پیامی از دوران پیش از طوفان باز آورد.</li><li>او راه دوری را آمد و خسته بود ولی به او استراحت داده شد،</li><li>همه (زحماتش) [بر] لوحی از سنگ نوشته شد.</li></ol><p>If you like this article please up vote to 50 claps and share with your friends and colleagues.</p><p>Author: <a href="https://github.com/Melanee-Melanee">Melanee</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f883e0ff068f" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-electronic-babylonian-library-ebl-gilgamesh-project-f883e0ff068f">The “electronic Babylonian Library” (eBL), Gilgamesh project</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monitor your public IP address using Python]]></title>
            <link>https://melanee-melanee.medium.com/monitor-your-public-ip-address-using-python-7956d18be9ec?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/7956d18be9ec</guid>
            <category><![CDATA[ip]]></category>
            <category><![CDATA[public-ip-address]]></category>
            <category><![CDATA[monitor]]></category>
            <category><![CDATA[vpn]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Tue, 30 May 2023 14:14:12 GMT</pubDate>
            <atom:updated>2023-05-30T14:15:34.001Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Monitor your public IP address using Python" src="https://cdn-images-1.medium.com/max/1024/0*amRz5szOSK1TmwHE" /><figcaption>Photo by <a href="https://unsplash.com/@sigmund?utm_source=medium&amp;utm_medium=referral">Sigmund</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A few days ago, one of my friends told me, he needs a program that can monitor his public IP to check if his VPN is connected or not.</p><p>He needs this program for his job. The company that he works for, calculates his salary according to how much he was connected to this VPN.</p><p>Finally, I came across this Python program for him:</p><pre><br>import requests   <br>import time<br><br>def get_public_ip():<br>    response = requests.get(&#39;https://api.ipify.org&#39;).text<br>    return response<br><br>interval = 5  # Interval in seconds<br>while True:<br>    public_ip = get_public_ip()<br>    print(f&#39;Public IP: {public_ip}&#39;)<br>    time.sleep(interval)</pre><p>These few lines of Python code can monitor your public IP every 5 seconds. If your public IP has been changed you can see the changes.</p><h3>Alternative uses</h3><p>There are number of use cases for this code, mine was checking if my computer is still connected to a VPN. Another use case is to implement this code in the back end of websites to check IP of clients and see where are they from and which country and language they use.</p><h3>Author:</h3><p><strong>Melanee</strong></p><p><a href="https://github.com/Melanee-Melanee">Melanee-Melanee - Overview</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7956d18be9ec" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Comprehensive Analysis of Hyperparameter Optimization in Logistic Regression Models]]></title>
            <link>https://levelup.gitconnected.com/a-comprehensive-analysis-of-hyperparameter-optimization-in-logistic-regression-models-521564c1bfc0?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/521564c1bfc0</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Sun, 21 May 2023 19:06:24 GMT</pubDate>
            <atom:updated>2023-05-21T19:06:24.990Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n4Dr2ttMux5XFLpr" /><figcaption>Photo by <a href="https://unsplash.com/@diane_soko?utm_source=medium&amp;utm_medium=referral">Diane Picchiottino</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3><strong><em>Introduction</em></strong></h3><p>In this article, we will follow a systematic approach to fine-tune the logistic regression algorithm’s hyperparameters. Our process will consist of the following steps:</p><blockquote><strong>1. Manually adjusting each hyperparameter<br>2. Evaluating the impact of each hyperparameter on the model’s accuracy<br>3. Identifying optimal values for the hyperparameters</strong></blockquote><p>Once we have determined the best hyperparameters, we will compare the performance of various models using these optimal values. Finally, we will provide a comprehensive analysis of the effectiveness of hyperparameter tuning in enhancing the model’s performance.</p><p>Before diving into our primary objective, it’s crucial to address some fundamental concepts related to this subject.</p><p><strong>Logistic Regression</strong></p><p>Logistic Regression is a supervised machine learning algorithm and is a linear model for binary classification. In this model, the probabilities that describe the possible outcomes of an experiment are modeled using a logistic function [1].</p><p>The <strong>logistic</strong> <strong>function</strong> is defined as:</p><blockquote><strong>f(x) = L / (1 + e^(-k(x-x0)))</strong></blockquote><p>where:</p><p>· <strong>x:</strong> the midpoint values of the sigmoid curve</p><p>· <strong>L:</strong> the maximum value of the curve</p><p>· <strong>k:</strong> the logistic growth rate or the slope of the curve</p><p><strong>Here</strong>, x represents the input features, and the output is a <strong>binary</strong> variable (0 or 1). The logistic regression algorithm learns the optimal values of the coefficients from the training data to predict the probability of the binary outcome for new input data. Once the model is trained, it can be used to predict the binary outcome by thresholding the predicted probability [1].</p><figure><img alt="Logistic regression equation" src="https://cdn-images-1.medium.com/max/387/1*5eXEK_WUBIMdftN3e-jGlQ.png" /><figcaption>Logistic Regression-Sigmoid Function [2]</figcaption></figure><p><strong>Parameters vs. Hyperparameters</strong></p><p>Model <strong>parameters</strong> and <strong>hyperparameters</strong> are the two parameter types used in Machine Learning (ML) models. Data estimation or data learning can be used to adjust the model’s parameters. <strong>Parameters</strong> are values that the model learns from the data, while <strong>hyperparameters</strong> are values that the user sets to control the behavior of the learning algorithm. <strong>Hyperparameters</strong> affect how the parameters are learned, and choosing good hyperparameters can be crucial for achieving good performance on a given task. However, the process of optimizing hyperparameters can be complex and time-consuming, especially when dealing with many hyperparameters or multiple optimization objectives. Therefore, it’s important to strike a balance between optimizing hyperparameters and avoiding overfitting the training data [3].</p><p><strong>Hyperparameter Tuning</strong></p><p>Hyperparameter tuning is<strong> </strong>a crucial part of the machine learning process as it can significantly affect the performance of the model. By selecting the right hyperparameters, such as learning rate and regularization, among others, a model can achieve optimal performance. However, the process of hyperparameter tuning can be challenging and time-consuming, as there are many hyperparameters to consider, and the search space can be vast. Therefore, it is essential to strike a balance between optimizing the model’s performance and avoiding an endless cycle of trying to optimize [3].</p><p><strong>Logistic Regression Hyperparameters</strong></p><p>The main hyperparameters we will tune in logistic regression are <strong>solver,</strong> <strong>penalty, max_iter, C (regularization strength), tol, fit_intercept, intercept_scaling, class_weight, random_state, multi_class, verbose, warm_start, and l1_ratio</strong> (<a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html">sklearn documentation</a>) [4].</p><p>In the optimization problem, the algorithm to use is called the <strong>solver</strong>. There are several choices for the solver, including <strong>‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’,</strong> and <strong>‘saga’</strong>. The <strong>default </strong>solver is <strong>‘lbfgs’</strong>.</p><ul><li>‘<strong>lbfgs</strong>’: This solver performs relatively well compared to other methods and saves a lot of memory. However, it may sometimes have issues with convergence.</li><li>‘<strong>sag</strong>’: This solver is faster than other solvers for large datasets, especially when both the number of samples and the number of features are large.</li><li>‘<strong>saga</strong>’: This solver is the preferred choice for sparse multinomial logistic regression and is suitable for very large datasets.</li><li>‘<strong>newton-cg</strong>’: This solver is computationally expensive due to the Hessian Matrix.</li><li>‘<strong>liblinear</strong>’: This solver is recommended for high-dimensional datasets, as it is effective in solving large-scale classification problems [4, 5].</li></ul><p><strong>Please note</strong> that “<strong><em>sag</em>”</strong> and “<strong><em>saga</em>”</strong> may only converge quickly when the features have a similar scale. To ensure this, we recommend scaling the data using either <strong>StandardScaler</strong> or <strong>MinMaxScaler</strong> from the sklearn.preprocessing module before fitting.</p><p><strong>The penalty (or regularization)</strong> aims to decrease the model’s generalization error and is designed to deter and control overfitting. This technique dissuades the learning of more complex models in order to prevent overfitting. Available options include: <strong>‘l1’, ‘l2’, ‘elasticnet’</strong>, and <strong>‘none’</strong>, with <strong>‘l2’</strong> being the default [4, 5].</p><p>However, not all penalties are compatible with every solver. The table below summarizes the supported penalties for each solver:</p><figure><img alt="Penalties supported by each solver" src="https://cdn-images-1.medium.com/max/816/1*2hqAhXTIiOFH_hSL7Yj1XA.jpeg" /><figcaption>Table taken from <a href="https://scikit-learn.org/dev/modules/linear_model.html#logistic-regression">Sckit-learn</a></figcaption></figure><h3>Results and discussion</h3><p>To accomplish our main task, we have chosen the <a href="https://www.kaggle.com/datasets/mathchi/diabetes-data-set">diabetes dataset</a> available on Kaggle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wplw1er9af7hdLYj" /><figcaption>Photo by <a href="https://unsplash.com/@mchesin?utm_source=medium&amp;utm_medium=referral">Matt C</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>The <strong>goal</strong> of this classification problem is to develop a model that can accurately classify patients as either <strong>diabetic</strong> or <strong>non-diabetic</strong> based on their medical information. This is a <strong>binary classification</strong> problem. This type of classification problem is important in healthcare as it can assist in the early diagnosis and treatment of diabetes, which is a chronic disease that affects millions of people worldwide.</p><p>The <strong>statistical overview</strong> of the dataset is presented below:</p><figure><img alt="Statistical details of the dataset" src="https://cdn-images-1.medium.com/max/1024/1*ncro0urywGt5IlKWdp0TTw.png" /><figcaption>Table taken from <a href="https://www.kaggle.com/datasets/mathchi/diabetes-data-set">Kaggle</a></figcaption></figure><p>Before tuning the hyperparameters, we create a <strong>baseline</strong> model by fitting the classifier with <strong>default</strong> parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dcf25177192f38d455ca53d1819b5e2e/href">https://medium.com/media/dcf25177192f38d455ca53d1819b5e2e/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d737d97993d24b9a3ff50d04339eb780/href">https://medium.com/media/d737d97993d24b9a3ff50d04339eb780/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2c969d16e9e042255513bab136644eeb/href">https://medium.com/media/2c969d16e9e042255513bab136644eeb/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0e8ab2b2dd2e25eecac7328da4e8992/href">https://medium.com/media/e0e8ab2b2dd2e25eecac7328da4e8992/href</a></iframe><p>Output:</p><figure><img alt="The result of the baseline model with default parameters" src="https://cdn-images-1.medium.com/max/597/1*N-E1RJ-V4uN1Nu5KVLGlcA.png" /></figure><p>We will use <strong>accuracy</strong>, <strong>precision</strong>, and<strong> recall</strong> as <strong>baseline metrics</strong> to compare any progress made during the optimization work. We will also examine the impact of each hyperparameter on model accuracy. In the <strong>primary model</strong>, default values were used for the solver and penalty hyperparameters.</p><p>Interestingly, the accuracy of the test data exceeded that of the train data. Moreover, we observed a precision of 0.73.</p><p><strong>Since</strong> the choice of solver depends on the chosen penalty, we aim to select the most suitable solver based on the penalties supported by each solver in order to enhance the metrics in comparison to the initial model.</p><p><strong><em>Finding the best ‘’solver” and “penalty”</em></strong></p><figure><img alt="The best “solver” and “penalty”" src="https://cdn-images-1.medium.com/max/398/1*fH-Z0KdgCDu5Fi_uqF7QUA.png" /><figcaption>Table by author</figcaption></figure><p><strong>At first</strong>, we start with the default value of the solver i.e. <strong>“lbfgs”</strong> and <strong>penalty = “None”</strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90e5a3f9df19d079d55552b613c684a8/href">https://medium.com/media/90e5a3f9df19d079d55552b613c684a8/href</a></iframe><p>Output:</p><figure><img alt="The results for solver = “lbfgs” and penalty = “None”" src="https://cdn-images-1.medium.com/max/595/1*qXuNg3K1D4Hbq8vOp9kF5Q.png" /></figure><p>It can be seen that the metrics of the model improved compared to the initial model.</p><p><strong>Then</strong> we calculate the metrics for the rest of the solvers with <strong>penalty = “None”</strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/67fe7dc7b9fcbd079a69c1cb0e793d4b/href">https://medium.com/media/67fe7dc7b9fcbd079a69c1cb0e793d4b/href</a></iframe><p>Output:</p><figure><img alt="The results for solver = {“newton-cg” , “sag”, “saga”}, and penalty = “None”" src="https://cdn-images-1.medium.com/max/488/1*sCjygM4yaE8pfkpJm8Zk2A.png" /></figure><p>Similarly, it can be seen that the metrics of the model improved compared to the initial model and obtained the same metrics as the previous setting (<strong>“lbfgs” </strong>and <strong>penalty = “None”</strong>).</p><p><strong>Now</strong>, we calculate metrics for solvers with <strong>penalty = ‘l2’</strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ebcb4b4504db97a0cf3e36e2be2ff879/href">https://medium.com/media/ebcb4b4504db97a0cf3e36e2be2ff879/href</a></iframe><p>Output:</p><figure><img alt="The results for solver = {“newton-cg” , “sag”, “saga”}, and penalty = “l2”" src="https://cdn-images-1.medium.com/max/487/1*cCIkryScYEdUhFXQFXjZuw.png" /></figure><p>It can be seen that these values of metrics are the same as the values of the base model.</p><p>By setting <strong>solver = “liblinear”</strong> with <strong>penalty = “l1/l2”</strong>, we calculate the metrics of the model:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/13d6cd4292ad13136f4f7803b1de3bbc/href">https://medium.com/media/13d6cd4292ad13136f4f7803b1de3bbc/href</a></iframe><p>Output:</p><figure><img alt="The results for solver = “liblinear” with penalty = “l1" src="https://cdn-images-1.medium.com/max/597/1*-diJ3_P_L77Ne0UIiypAMw.png" /></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0c2a4c7c3593ce8e996578ab9263b62/href">https://medium.com/media/b0c2a4c7c3593ce8e996578ab9263b62/href</a></iframe><p>Output:</p><figure><img alt="The results for solver = “liblinear” with penalty = “l2”" src="https://cdn-images-1.medium.com/max/597/1*JsWX4C4ahB9232NRvseTAw.png" /></figure><p>It is evident that the training and testing accuracies for the <strong>L1</strong> penalty are superior to those for the <strong>L2</strong> penalty. Conversely, the L1 penalty’s precision value is not as high as that of the L2 penalty.</p><p>The <strong>optimal penalties</strong> for enhancing model accuracy for various solvers, based on the investigations conducted on the diabetes dataset, are displayed in the table below:</p><figure><img alt="Optimal penalties for various solvers" src="https://cdn-images-1.medium.com/max/312/1*S_09Ci5C7oFhToQacf-lfQ.png" /><figcaption>Table by author</figcaption></figure><p>In the following, we assess the impact of various hyperparameters on the model’s accuracy to determine if they enhance its performance or not.</p><p><strong><em>Examining “max_iter” parameter</em></strong></p><p>In the following step, we incorporate a <strong>max_iter</strong> layer to assess its impact on the model’s accuracy. Typically, we employ the default max_iter value, which is <strong>max_iter = 100</strong>. Alternatively, we can select the maximum iteration count using the max_iter parameter, where we usually increase the value for convergence when we have an extensive amount of train data [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/955acc9cb6385ec6d12fbba5d8455241/href">https://medium.com/media/955acc9cb6385ec6d12fbba5d8455241/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “max_iter”" src="https://cdn-images-1.medium.com/max/482/1*_0KDG7A5u2Vhk7RrC-rRxA.png" /></figure><p>It is evident that when varying the <strong>max_iter</strong> with values of 100, 200, 500, and 1000 did not lead to any significant changes in the model’s test accuracy.</p><p><strong><em>Examining “C” parameter</em></strong></p><p><strong>“C”</strong> (in the default mode is equal to 1) is a penalty term that controls the trade-off between the complexity and accuracy of a machine learning model to eliminate and adjust overfitting. A <strong>low </strong>value of “C” means that the model prefers simplicity overfitting the data well. This may lead to underfitting or misclassification of some examples. A <strong>high </strong>value of “C” means that the model tries to fit the data as well as possible, even if it means using a more complex decision boundary. This may lead to overfitting or sensitivity to noise. (A <strong>high </strong>“C” value indicates that the model gives more weight to the training data and reduces the weight assigned to the complexity penalty, and a <strong>low </strong>value of “C” indicates the model gives more weight to this complexity penalty at the cost of fitting the training data). <strong>{<em>float, default=1.0</em>}</strong> [4].</p><p>We aim to examine the impact of balancing the training data against the complexity penalty. To achieve this, we will set the penalty type to “<strong>l2</strong>”.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9113f20a2061ac24e7c4797c079e99e3/href">https://medium.com/media/9113f20a2061ac24e7c4797c079e99e3/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “C”" src="https://cdn-images-1.medium.com/max/491/1*MH0p9ipRBhg8MW9Bgqg_EQ.png" /></figure><p>It appears that when the values of <strong>C</strong> are adjusted from <strong>1 to 10</strong>, the model’s accuracy improves as C increases. This effectively assigns more weight to the training data compared to the complexity penalty. Furthermore, assigning a value of C higher than 10 did not result in any additional improvement in test accuracy.</p><p><strong><em>Examining “tol” parameter</em></strong></p><p>The parameter <strong>“tol”</strong> controls the tolerance for the stopping criterion of the algorithm. A <strong>larger value</strong> means the algorithm will stop sooner, but it may not reach the optimal solution. A <strong>smaller value</strong> means the algorithm will run longer, but it may find a better solution. However, there is a trade-off between accuracy and efficiency. <strong>{<em>float, default=1e-4</em>}<em> </em></strong>[4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbb842058db7305a149b54c8794decaa/href">https://medium.com/media/cbb842058db7305a149b54c8794decaa/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “tol”" src="https://cdn-images-1.medium.com/max/490/1*LROiOyuXj2y9KEKTTIpvuA.png" /></figure><p>As the value of <strong>tol</strong> rises from 0.0001 (the <strong>default</strong> value) to 10, the test accuracy diminishes. Lower values allow the algorithm more time to converge.</p><p><strong><em>Examining “fit_intercept” parameter</em></strong></p><p>The parameter <strong>“fit_intercept”</strong> controls whether the logistic regression model has a bias term or not. If we set <strong>fit_intercept = False</strong>, the model assumes that the data is centered around the origin, and the equation of the decision boundary will have a zero bias. The default value of <strong>“fit_intercept” </strong>is <strong>True</strong>, which means that the model will learn a non-zero bias from the data [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5adcab549e187fa79f7fa484a4b90393/href">https://medium.com/media/5adcab549e187fa79f7fa484a4b90393/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “fit_intercept”" src="https://cdn-images-1.medium.com/max/492/1*CAxxmKu503q_W0yS3HStMA.png" /></figure><p>The results show that setting <strong>fit_intercept = True</strong> provides <strong>better </strong>test<strong> </strong>accuracy than setting <strong>fit_intercept = False</strong>.</p><p><strong><em>Examining “intercept_scaling” parameter</em></strong></p><p>The <strong>“intercept_scaling”</strong> parameter is used to reduce the effect of regularization on the intercept term. By increasing the “intercept_scaling” value, you can indeed reduce the regularization effect on the feature weight and the decision boundary width. <strong>{<em>float, default=1</em>}</strong> [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/18dbc3488ddf51d7c96f2ad87873bc07/href">https://medium.com/media/18dbc3488ddf51d7c96f2ad87873bc07/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “intercept_scaling”" src="https://cdn-images-1.medium.com/max/490/1*Mg1jFstzwFdou7Uo9S6TVw.png" /></figure><p>Analyzing the outcomes of different <strong>intercept_scaling</strong> values shows that the <strong>optimal</strong> accuracy is achieved for 0.2.</p><p><strong><em>Examining “class_weight” parameter</em></strong></p><figure><img alt="Class distribution of the diabetes dataset" src="https://cdn-images-1.medium.com/max/490/1*1i20pvTkIl5OHhD6Rp69mQ.jpeg" /><figcaption>Photo by author</figcaption></figure><p>The figure above shows that the class distribution of the diabetes dataset is <strong>imbalanced</strong>.</p><p>The parameter <strong>“class_weight”</strong> adjusts the weight of each class in logistic regression. If the data has <strong>imbalanced </strong>classes, we can set <strong>class_weight = ‘balanced’</strong>, which means that the model will use inverse proportions of the class frequencies to balance the data. <strong>{<em>dict or ‘balanced’, default=None</em>}</strong> [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8558ab5757c8525b21782fb20a26f301/href">https://medium.com/media/8558ab5757c8525b21782fb20a26f301/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “class_weight”" src="https://cdn-images-1.medium.com/max/490/1*RgZcRDSbTcDMzfbosWTtjg.png" /></figure><p>The results show that the <strong>‘None’</strong> mode exhibited greater test accuracy than the balanced mode.</p><p><strong><em>Examining “random_state” parameter</em></strong></p><p>The parameter <strong>“random_state”</strong> sets the seed for the random number generator in logistic regression. This ensures that the algorithm has a consistent randomness, which can be reproduced by using the same value of random_state. <strong>{<em>int, RandomState instance, default=None</em>}</strong> [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4bb4ec0a9a0aa9eb8acd3ff3799b2f01/href">https://medium.com/media/4bb4ec0a9a0aa9eb8acd3ff3799b2f01/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “random_state”" src="https://cdn-images-1.medium.com/max/491/1*fAYjFnmxtuMwO8faez3N4g.png" /></figure><p>No enhancement in model test accuracy was noticed when altering the values of random_state.</p><p><strong><em>Examining “multi_class” parameter</em></strong></p><p>The parameter <strong>“multi_class”</strong> determines how to handle multiple classes in logistic regression. If it is <strong>‘ovr’</strong>, the model will use a <strong>one-vs-rest strategy</strong>, which means fitting a binary model for each class. If it is <strong>‘multinomial’</strong>, the model will use the <strong>softmax function</strong>, which means minimizing the multinomial loss over the whole probability distribution. The <strong>‘multinomial’</strong> option is not compatible with <strong>solver = ‘liblinear’</strong>. The default value of “multi_class” depends on the data and the solver. If the data is binary or the solver is ‘liblinear’, it will be ‘ovr’. Otherwise, it will be ‘multinomial’. <strong><em>{‘auto’, ‘ovr’, ‘multinomial’}, default=’auto’</em></strong> [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5858a44b0938128b745cd11a19c7af7e/href">https://medium.com/media/5858a44b0938128b745cd11a19c7af7e/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “multi_class”" src="https://cdn-images-1.medium.com/max/490/1*F0lR0N39cCu34H453UJUlA.png" /></figure><p>The <strong>comparison</strong> of <strong>“multi_class”</strong> results is given in the following table:</p><figure><img alt="Comparison solvers" src="https://cdn-images-1.medium.com/max/434/1*Ow_6MAtG2p3_9wENnOgoGQ.png" /><figcaption>Table by author</figcaption></figure><p><strong><em>Examining “verbose” parameter</em></strong></p><p>The parameter <strong>“verbose”</strong> controls the verbosity of the algorithm. It determines how many messages are displayed during the model optimization. It can be set to any positive integer value. A <strong>larger value</strong> means more messages are shown. The “verbose” option is only available for <strong>solver = ‘liblinear’</strong>, and <strong>‘lbfgs’</strong> [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/983744a67083c08e4d2609103b79fa86/href">https://medium.com/media/983744a67083c08e4d2609103b79fa86/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “verbose” {0.1–10}" src="https://cdn-images-1.medium.com/max/490/1*t3LKBtFKjjrUKjc_2aPbiQ.png" /></figure><p>The results indicate that varying values of <strong>“verbose”</strong> had no impact on enhancing the model’s test accuracy.</p><p><strong><em>Examining “warm_start” parameter</em></strong></p><p>The parameter <strong>“warm_start”</strong> allows us to use the previous model’s solution as the starting point for the current model. This can speed up convergence and save computation time. The default value of <strong>“warm_start”</strong> is <strong>False</strong>, which means that the model will start from scratch each time. We can set <strong>warm_start = True</strong> if we want to reuse the previous model’s learnings. Moreover, it is useless for the <strong>“liblinear”</strong> solver [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/311224e57433b6df56788dd2778a23c3/href">https://medium.com/media/311224e57433b6df56788dd2778a23c3/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “warm_start” of different solvers" src="https://cdn-images-1.medium.com/max/490/1*mWWFbgz1YIAxLzr8_bzDsA.png" /></figure><p>The test accuracy of the model was not affected by changing the <strong>“warm_start”</strong> to <strong>True </strong>or<strong> False</strong> for different solvers.</p><p><strong><em>Examining “l1_ratio” parameter</em></strong></p><p>The parameter <strong>“l1_ratio”</strong> controls the balance between ‘l1’ and ‘l2’ regularization in logistic regression.</p><p>· Setting <strong>l1_ratio = 0</strong> is equivalent to using <strong>penalty = ‘l2’</strong>.</p><p>· Setting <strong>l1_ratio = 1</strong> is equivalent to using <strong>penalty = ‘l1’</strong>.</p><p>· If <strong>0 &lt; l1_ratio &lt; 1</strong>, the model will use a <strong>mix</strong> of <strong>l1 </strong>and<strong> l2</strong> penalties.</p><p>The parameter <strong>“l1_ratio”</strong> is only applicable when <strong>penalty = “elasticnet”</strong> and <strong>solver= “saga”</strong> in logistic regression [4].</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08508beb6c9054224b9b3e04b4b75b4f/href">https://medium.com/media/08508beb6c9054224b9b3e04b4b75b4f/href</a></iframe><p>Output:</p><figure><img alt="Comparing metrics for different “l1_ratio” {0, 0.5, 1}" src="https://cdn-images-1.medium.com/max/491/1*OGYkmbmANZlV49dSmUcnBQ.png" /></figure><p>The <strong>comparison</strong> of the results shows that higher accuracy is provided for <strong>l1_ratio=1 &gt; l1_ratio=0.5 &gt; l1_ratio=0</strong>, respectively.</p><p><strong>The best hyperparameters</strong> for the diabetes dataset were extracted from the above results and <strong>summarized</strong> in the table below:</p><figure><img alt="The optimum hyperparameters for the diabetes dataset" src="https://cdn-images-1.medium.com/max/616/1*SYtA97au5Npo-0dmcKU-bw.png" /><figcaption>Table by author</figcaption></figure><p><strong>In the end</strong>, we make several models according to the above summary and compare the results with each other:</p><p><strong>Model 1: Default hyperparameters</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/734096b5a3a835b9ce10dbadcc0a582a/href">https://medium.com/media/734096b5a3a835b9ce10dbadcc0a582a/href</a></iframe><p>Output:</p><figure><img alt="Accuracy of train and testing set for model 1" src="https://cdn-images-1.medium.com/max/596/1*a1bQeeh9dyRW6XOd8ryTKw.png" /></figure><figure><img alt="Confusion matrix of train and test set for model 1" src="https://cdn-images-1.medium.com/max/1024/1*BcxcCkyMEvD5MFyMJ-6GBA.png" /><figcaption>Confusion Matrix (Model 1)</figcaption></figure><figure><img alt="Classification report on test set for model 1" src="https://cdn-images-1.medium.com/max/518/1*5DvPS2g-mdyMIFniTgPetw.png" /><figcaption>Classification Report (Model 1)</figcaption></figure><figure><img alt="Model curves for model 1" src="https://cdn-images-1.medium.com/max/1024/1*jZJYkthqxgPPVP0Ujozu8A.png" /><figcaption>Model Curves (Model 1)</figcaption></figure><p><strong>Model 2: Best hyperparameters for “<em>liblinear</em>” (Manually tuned)</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dcc781ed9326343eb3f64b85a112fd5b/href">https://medium.com/media/dcc781ed9326343eb3f64b85a112fd5b/href</a></iframe><p>Output:</p><figure><img alt="Accuracy of train and testing set for model 2" src="https://cdn-images-1.medium.com/max/592/1*WVrMArcSX9F4KHWjOYBP0g.png" /></figure><figure><img alt="Confusion matrix of train and test set for model 2" src="https://cdn-images-1.medium.com/max/1024/1*qXbRaPqnhAMHPkHL4dPcxQ.png" /><figcaption>Confusion Matrix (Model 2)</figcaption></figure><figure><img alt="Classification report on test set for model 2" src="https://cdn-images-1.medium.com/max/522/1*5ppaGDFIkz7WaSPtRvpawg.png" /><figcaption>Classification Report (Model 2)</figcaption></figure><figure><img alt="Model curves for model 2" src="https://cdn-images-1.medium.com/max/1024/1*4qzCLXWsLbS73u1riVReEw.png" /><figcaption>Model Curves (Model 2)</figcaption></figure><p><strong>Model 3: Best hyperparameters for “<em>lbfgs</em>” (Manually tuned)</strong></p><p>It is important to mention that in <strong>Model 3</strong>, the default value was used for <strong>“C”</strong>. If this value was not set accordingly, the model would experience an error.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/20e6eeb629a17b76e09b335f2df6be8d/href">https://medium.com/media/20e6eeb629a17b76e09b335f2df6be8d/href</a></iframe><p>Output:</p><figure><img alt="Accuracy of train and testing set for model 3" src="https://cdn-images-1.medium.com/max/595/1*pvdKYUtu0kycMKMpltyUew.png" /></figure><figure><img alt="Confusion matrix of train and test set for model 3" src="https://cdn-images-1.medium.com/max/1024/1*N32p3YJJuzZl6pqCsNdU_A.png" /><figcaption>Confusion Matrix (Model 3)</figcaption></figure><figure><img alt="Classification report on test set for model 3" src="https://cdn-images-1.medium.com/max/522/1*2wQscIJRz_CZ5VWUajzZ2g.png" /><figcaption>Classification Report (Model 3)</figcaption></figure><figure><img alt="Model curves for model 3" src="https://cdn-images-1.medium.com/max/1024/1*I-Zq5jjKuq-v_Me8F6anwg.png" /><figcaption>Model Curves (Model 3)</figcaption></figure><p><strong>Model 4: Best hyperparameters for “<em>newton-cg</em>” (Manually tuned)</strong></p><p>It is important to mention that for <strong>Model 4</strong>, the value of <strong>“C”</strong> was kept at its default setting; otherwise, the model would experience an error.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d81668973ad1065a09b561833e4c113d/href">https://medium.com/media/d81668973ad1065a09b561833e4c113d/href</a></iframe><p>Output:</p><figure><img alt="Accuracy of train and testing set for model 4" src="https://cdn-images-1.medium.com/max/595/1*-xniAD3jyfOhKU65mE9Wiw.png" /></figure><figure><img alt="Confusion matrix of train and test set for model 4" src="https://cdn-images-1.medium.com/max/927/1*mKM5v13fBFwA3iXZJBHvBA.png" /><figcaption>Confusion Matrix (Model 4)</figcaption></figure><figure><img alt="Classification report on test set for model 4" src="https://cdn-images-1.medium.com/max/522/1*WIS1Pq0Y6ng3z9ie__hzfw.png" /><figcaption>Classification Report (Model 4)</figcaption></figure><figure><img alt="Model curves for model 4" src="https://cdn-images-1.medium.com/max/937/1*gVAu1BaTcWScDW_TjEGS9A.png" /><figcaption>Model Curves (Model 4)</figcaption></figure><p>The <strong>results</strong> of the models (1–4) are summarized in the following table:</p><figure><img alt="The results of the models 1–4" src="https://cdn-images-1.medium.com/max/1024/1*yD7bQnz8dYISEtUgkzCjoA.png" /><figcaption>Table by author</figcaption></figure><p>Moreover, as we mentioned earlier in this article, <strong>manual</strong> hyperparameter tuning can be a time-consuming process. However, there are several automated methods available, such as <strong>GridSearchCV</strong> and <strong>RandomizedSearchCV</strong>, which are provided by Scikit-learn [6, 7].</p><h3><strong>Conclusion</strong></h3><p>The manually tuned <strong>Models 2</strong> through <strong>4</strong> exhibited improved performance in terms of <strong>accuracy</strong>, <strong>precision</strong>, <strong>recall</strong>, and <strong>F1 score</strong> on the <strong>test</strong> set in comparison to <strong>Model 1</strong>, which utilized default hyperparameters. Moreover, the classifiers in <strong>Models 2</strong> to <strong>4</strong> demonstrated enhanced performance in predicting label <strong>1</strong> (with six more correct labels) compared to<strong> Model 1</strong>. This suggests that the classifiers in <strong>Models 2</strong> to <strong>4</strong> were more effective in predicting <strong>total correct labels</strong> than the classifier in <strong>Model 1</strong> with the default hyperparameters.</p><p><strong>In summary</strong>, understanding the influence of hyperparameters on a model’s performance and becoming proficient in adjusting them is essential for achieving optimal results.</p><p>We hope our findings have provided you with a deeper understanding of this crucial aspect of machine learning.</p><p><strong>The source code of this article is available on </strong><a href="https://github.com/alifrmf/Hyperparameter-Optimization"><strong>GitHub</strong></a><strong>.</strong></p><h3><strong>References</strong></h3><p>[1] <a href="https://www.geeksforgeeks.org/understanding-logistic-regression/">https://www.geeksforgeeks.org/understanding-logistic-regression/</a></p><p>[2] <a href="https://www.spiceworks.com/tech/artificial-intelligence/articles/what-is-logistic-regression/">https://www.spiceworks.com/tech/artificial-intelligence/articles/what-is-logistic-regression/</a></p><p>[3] <a href="https://www.blog.trainindata.com/hyperparameter-tuning-for-machine-learning/">https://www.blog.trainindata.com/hyperparameter-tuning-for-machine-learning/</a></p><p>[4] <a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html">https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html</a></p><p>[5] <a href="https://medium.com/codex/do-i-need-to-tune-logistic-regression-hyperparameters-1cb2b81fca69">https://medium.com/codex/do-i-need-to-tune-logistic-regression-hyperparameters-1cb2b81fca69</a></p><p>[6] <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html">https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html</a></p><p>[7] <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html">https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html</a></p><h3><strong>Author: Ali Farahmandfar</strong></h3><p><a href="https://github.com/alifrmf">alifrmf - Overview</a></p><h3>Editor: Melanee</h3><p><a href="https://github.com/Melanee-Melanee">Melanee-Melanee - Overview</a></p><h3>Level Up Coding</h3><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://levelup.gitconnected.com/?utm_source=pub&amp;utm_medium=post">Level Up Coding publication</a></li><li>💰 Free coding interview course ⇒ <a href="https://skilled.dev/?utm_source=luc&amp;utm_medium=article">View Course</a></li><li>🔔 Follow us: <a href="https://twitter.com/gitconnected">Twitter</a> | <a href="https://www.linkedin.com/company/gitconnected">LinkedIn</a> | <a href="https://newsletter.levelup.dev/">Newsletter</a></li></ul><p>🚀👉 <a href="https://jobs.levelup.dev/talent/welcome?referral=true"><strong>Join the Level Up talent collective and find an amazing job</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=521564c1bfc0" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/a-comprehensive-analysis-of-hyperparameter-optimization-in-logistic-regression-models-521564c1bfc0">A Comprehensive Analysis of Hyperparameter Optimization in Logistic Regression Models</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI developers optimize and reduce execution time for Python ML models?]]></title>
            <link>https://melanee-melanee.medium.com/how-ai-developers-optimize-and-reduce-execution-time-for-python-ml-models-1087cef3194f?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/1087cef3194f</guid>
            <category><![CDATA[optimization-algorithms]]></category>
            <category><![CDATA[execution-time]]></category>
            <category><![CDATA[time-complexity]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Fri, 28 Apr 2023 19:20:00 GMT</pubDate>
            <atom:updated>2023-04-28T20:16:57.469Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="How an AI developer optimize and reduce execution time for python ML models?" src="https://cdn-images-1.medium.com/max/1024/0*2jcL1R8kT5IC_7Rl" /><figcaption>Photo by <a href="https://unsplash.com/@agebarros?utm_source=medium&amp;utm_medium=referral">Agê Barros</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>As a programmer you must know that Python is an <a href="https://www.freecodecamp.org/news/compiled-versus-interpreted-languages/">interpreter</a> programming language and these sorts of programming languages are slow in comparison to <a href="https://www.geeksforgeeks.org/compiler-vs-interpreter-2/">compiler</a> programming languages like Java and C++. But there are some strategies that artificial intelligence(AI) developers can implement to optimize and decrease execution time for Python machine learning (ML) models, for instance:</p><h4>Using binary formats for saving models</h4><p>Saving machine learning models in binary formats like .pkl, .H5, or .pb can decrease execution time for Python.</p><h4>Using JIT compilers</h4><p>Just-In-Time (JIT) compilers can compile Python code on the fly, which can improve the performance of the code [1].</p><h4>Vectorization</h4><p>This technique involves performing operations on arrays and matrices instead of looping through each element. This can significantly enhance the performance of the code. AI developer can use NumPy or pandas libraries for vectorization. They can reduce the number of iterations required and speed up computations.</p><h4>Parallelization</h4><p>Parallel processing can be used to distribute the workload across multiple CPUs or GPUs. This can result in significant speedups for certain operations. Parallelization is a technique of executing multiple tasks simultaneously. You can use libraries like Joblib or Dask to parallelize the execution of machine learning models. It involves dividing a computation into smaller parts that can be run in parallel on multiple cores or CPUs [2].</p><h4>Algorithm optimization</h4><p>The choice of algorithm can have a significant impact on the performance of the code. AI developers can optimize the algorithms used in their machine learning models to make them more efficient. Some algorithms perform better than others on certain types of data. You can use libraries like scikit-learn or TensorFlow to try different algorithms and choose the best one for your data [3].</p><h4>Hardware acceleration</h4><p>Hardware acceleration techniques such as using GPUs or TPUs can remarkably speed up the training of machine learning models. You can use cloud computing services like AWS, Google Cloud Platform or Microsoft Azure to use hardware with better performance.</p><h4>Using optimized libraries</h4><p>Python has several libraries such as NumPy, SciPy, and Pandas that are optimized for scientific computing and machine learning. These libraries provide highly optimized algorithms for common operations and can significantly speed up the code[4].</p><h4>Data preprocessing</h4><p>Preprocess your data to remove any unwanted data and transform the data into a format that is easier for the machine learning model to use. This can help to speed up the execution.</p><h4>Code optimization</h4><p>AI developers can optimize their code by identifying <a href="https://en.wikipedia.org/wiki/Bottleneck_(software)">bottlenecks</a> and optimizing critical sections of the code. This can involve rewriting code in a lower-level language, reducing memory usage, or optimizing algorithms. AI developer can write optimized code that takes advantage of the strengths of Python and use libraries like Cython or Numba to optimize Python code.</p><h4>Using pre-trained models</h4><p>Pre-trained models can save time and computational resources by providing a pre-trained network that can be fine-tuned for a specific task.</p><h3>Are binary formats faster?</h3><figure><img alt="Are binary formats faster?" src="https://cdn-images-1.medium.com/max/777/1*vVa2KlXIDPPArLt2fbHMsg.jpeg" /><figcaption>Photo by <a href="https://depositphotos.com/32429005/stock-photo-01-binary-code-on-led.html">depositphotos</a></figcaption></figure><p>Binary formats in ml models like .pkl, .H5, and .pb are file formats used to store data and machine learning models in a binary format, which is a compact, binary representation of the data that is faster to read and write compared to text-based formats like .py or .csv.</p><ul><li>.pkl is the file format used by Python’s pickle library to serialize and deserialize Python objects, including machine learning models [5].</li><li>.H5 is the file format used by the Hierarchical Data Format (HDF) library to store large and complex datasets, including machine learning models.</li><li>.pb is the file format used by Google’s Protocol Buffers to serialize and deserialize structured data, including machine learning models.</li></ul><p>Using binary formats to store machine learning models is often preferred over other formats because they are more compact and efficient to read and write, and can be easily loaded into memory for use in training or inference.</p><h3>Summary</h3><p>while Python may not be as fast as some other programming languages, there are several techniques that AI developers can use to optimize and decrease the executing time in their machine learning models. By means of these techniques, Python is still the most applied programming language for developing AI applications.</p><h3>References</h3><p>[1] <a href="https://www.ibm.com/docs/en/sdk-java-technology/8?topic=reference-jit-compiler">https://www.ibm.com/docs/en/sdk-java-technology/8?topic=reference-jit-compiler</a></p><p>[2] <a href="https://joblib.readthedocs.io/en/latest/">https://joblib.readthedocs.io/en/latest/</a></p><p>[3] <a href="https://medium.com/mlearning-ai/how-do-i-choose-a-machine-learning-algorithm-for-my-application-1699967bb31a">https://medium.com/mlearning-ai/how-do-i-choose-a-machine-learning-algorithm-for-my-application-1699967bb31a</a></p><p>[4] <a href="https://medium.com/geekculture/make-python-run-faster-c73137598bae">https://medium.com/geekculture/make-python-run-faster-c73137598bae</a></p><p>[5] <a href="https://www.datacamp.com/tutorial/pickle-python-tutorial">https://www.datacamp.com/tutorial/pickle-python-tutorial</a></p><h3>Author: Melanee</h3><p><a href="https://github.com/Melanee-Melanee">Melanee-Melanee - Overview</a></p><h4><a href="https://mlearning.substack.com/about">BECOME a WRITER at MLearning.ai</a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1087cef3194f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What a data scientist should know about machine learning kernels?]]></title>
            <link>https://melanee-melanee.medium.com/what-a-data-scientist-should-know-about-machine-learning-kernels-4066f02d1f26?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/4066f02d1f26</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[kernel]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[kernel-trick]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Fri, 14 Apr 2023 06:18:58 GMT</pubDate>
            <atom:updated>2023-04-14T16:41:34.942Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="machine learning kernels" src="https://cdn-images-1.medium.com/max/1024/0*Pm9NbxAYf80Bayu2" /><figcaption>Photo by <a href="https://unsplash.com/@robowunderkind?utm_source=medium&amp;utm_medium=referral">Robo Wunderkind</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>In general</strong>, a data scientist should have a basic understanding of the following concepts related to kernels in machine learning:</p><p><strong>1. What are kernels?</strong></p><p><strong>2. Types of kernels.</strong></p><p><strong>3. Purpose of kernels.</strong></p><p><strong>4. Selection of kernels.</strong></p><p><strong>5. Hyperparameter tuning.</strong></p><p><strong>6. Limitations of kernels.</strong></p><p>Before we discuss the above related to kernels in machine learning, let’s first go over a few basic concepts: <strong>Support Vector Machine</strong>, S<strong>upport Vectors </strong>and <strong>Linearly vs. Non-linearly Separable Data</strong>.</p><h4><strong>Support Vector Machine</strong></h4><p>Support Vector Machine (<strong>SVM</strong>) is a supervised learning algorithm used for classification and regression analysis. The main idea behind <strong>SVM</strong> is to find a hyperplane that separates the data into different classes with the maximum margin. The hyperplane is chosen in such a way that it maximizes the distance between the closest data points (called support vectors) from either class [1].</p><figure><img alt="Support Vector Machine" src="https://cdn-images-1.medium.com/max/440/1*ormKhw8YnQMwxEVdLuIodQ.png" /><figcaption><em>Figure. 1. Data Split by Decision Boundaries [2]</em></figcaption></figure><figure><img alt="Support Vector Machine" src="https://cdn-images-1.medium.com/max/528/1*r18AuCj1qn7WeQGg_qsC5Q.png" /><figcaption><em>Figure. 2. Margin and Maximum Margin Classifier [2]</em></figcaption></figure><h4><strong>Support Vectors</strong></h4><p>Support Vectors are the data points that lie closest to the hyperplane and are the most critical in defining the hyperplane. If the support vectors are changed, the hyperplane will also change, so support vectors play a crucial role in <strong>SVM</strong>. These support vectors determine the boundary between the classes and, as a result, have the greatest impact on the <a href="https://www.iguazio.com/glossary/model-accuracy-in-ml/">accuracy</a> of the model [1].</p><figure><img alt="Support Vectors" src="https://cdn-images-1.medium.com/max/477/1*P30AElJmPSDhnm1yTnerlA.png" /><figcaption><em>Figure. 3. The points closest to the decision boundary are called support vectors. SVM finds the decision boundary by maximizing its distance from the Support Vectors [3].</em></figcaption></figure><h4><strong>Linearly vs. Non-linearly Separable Data</strong></h4><p><strong>Linearly</strong> separable data is data that can be separated into different classes or categories by a single straight line, or hyperplane, in the feature space. That is, the data points belonging to different classes can be separated by a line in such a way that there are no points from different classes on the same side of the line. In other words, the data points can be perfectly separated by a linear boundary [4].</p><p>On the other hand, <strong>non-linearly</strong> separable data is data that cannot be separated into different classes by a single straight line or hyperplane in the feature space. This means that the data points belonging to different classes overlap in the feature space and a linear boundary cannot perfectly separate the data points into different classes. In these cases, a non-linear boundary is required to separate the data into different classes [2].</p><p><strong>For example</strong>, consider a dataset containing two classes of data points, “A” and “B”. If the data points can be separated into two classes by a single straight line, then the data is linearly separable. If the data points cannot be separated into two classes by a single straight line, then the data is non-linearly separable.</p><figure><img alt="Linearly Separable and B: Non-linearly Separable Data" src="https://cdn-images-1.medium.com/max/587/1*9ETgJNMHOA3tjtLgQXd_HQ.png" /><figcaption><em>Figure. 4. A: Linearly Separable and B: Non-linearly Separable Data [5]</em></figcaption></figure><p>After comprehending the above concepts, it is now easier to relate and understand the Kernels.</p><h3><strong>What are kernels?</strong></h3><p>Machine learning algorithms rely on mathematical functions called “kernels” to make predictions based on input data. A kernel is a mathematical function that maps input data into a higher-dimensional space, where patterns are easier to identify and classify [6].</p><p><strong>Kernel Methods:</strong></p><p>Kernel methods are a class of machine learning algorithms that make use of kernels to transform the input data into a higher-dimensional space. The transformed data is then used to make predictions about new data or to separate data into different classes. Kernel methods are particularly useful for data that is not linearly separable in its original space. Examples of kernel methods include <strong>SVM</strong>, kernel principal component analysis (<strong>PCA</strong>), and kernel density estimation [7–9].</p><p>In summary, kernels and kernel methods are important concepts in <a href="https://en.wikipedia.org/wiki/Machine_learning">machine learning</a>, particularly for problems involving non-linear relationships between variables. By transforming the input data into a higher-dimensional space, kernels allow machine learning algorithms to capture and model non-linear relationships, improving accuracy and performance.</p><p><strong>The Kernel Trick:</strong></p><p>The kernel trick is a technique used in kernel methods to avoid the need to explicitly compute the mapping of the input data into a higher-dimensional space. Instead of explicitly computing the mapping, the kernel trick uses the dot product between the input data in the original space to calculate the dot product between the mapped data in the higher-dimensional space. This allows the computation to be performed more efficiently, without the need to compute explicit mapping [10–12].</p><figure><img alt="Non-linear classifier using Kernel trick" src="https://cdn-images-1.medium.com/max/1024/1*GGvUEKtzWmtCTwQe67W37g.png" /><figcaption><em>Figure. 5. Non-linear classifier using Kernel trick [13]</em></figcaption></figure><p>The mathematics of the kernel trick is based on the concept of reproducing kernel Hilbert spaces (<strong>RKHS</strong>). An <strong>RKHS</strong> is a Hilbert space that is completely characterized by its inner product, also known as its kernel function. In machine learning, the kernel function maps the input data into a high-dimensional feature space, where a linear boundary can be used to separate the data into different classes.</p><p>The key mathematical idea behind the kernel trick is that the dot product between the mapped data in the high-dimensional feature space can be expressed in terms of the dot product between the input data in the original space. This is achieved through the use of a positive definite kernel function, k(x,y), which satisfies the property:</p><p><strong><em>k(x,y) = &lt;φ(x), φ(y)&gt;</em></strong></p><p>where φ(x) is the mapping of the input data into the high-dimensional feature space and &lt;.,.&gt; is the inner product in the RKHS.</p><p>By using the kernel trick, the need to compute the explicit mapping φ(x) is avoided, and the dot product between the mapped data in the high-dimensional feature space can be calculated using only the dot product between the input data in the original space. This leads to significant computational savings and improved performance for kernel methods [14–17].</p><h3><strong>Types of kernels</strong></h3><p>There are many types of kernels that can be used in machine learning, including:</p><p><strong>1. Linear Kernels</strong></p><p><strong>2. Polynomial Kernels</strong></p><p><strong>3. Gaussian Kernels (Radial Basis Function)</strong></p><p><strong>4. Sigmoid Kernels</strong></p><p><strong>5. Laplacian Kernels</strong></p><p><strong>6. Cosine Similarity Kernels</strong></p><p><strong>7. Histogram Intersection Kernels</strong></p><p><strong>8. Hyperbolic Kernels</strong></p><p><strong>In this article</strong>, we will discuss the different types of kernels used in machine learning with the mathematics behind them by an example and the scenarios where each is commonly used.</p><h3><strong>1.</strong> <strong>Linear Kernels</strong></h3><p>Linear kernels are the simplest type of kernel, used primarily for linear classification problems. A linear kernel maps the input data into a higher-dimensional space by multiplying the input by a weight matrix. The resulting output is then passed through a linear activation function, which separates the data into two or more classes based on a decision boundary. The linear kernel is ideal for linear problems, such as logistic regression or support vector machines (<strong>SVMs</strong>).</p><p>The linear kernel is suitable for use when the data is linearly separable. We can express that a linear kernel is a similarity function that measures the dot product between two vectors in a high-dimensional feature space. Mathematically, it can be expressed as:</p><p><strong><em>K(x, y) = &lt;x, y&gt;</em></strong></p><p>where &lt;x, y&gt; denotes the dot product of vectors x and y, and K(x, y) is the similarity score between x and y [18].</p><p><strong>For example</strong>, let’s say we have two 2-dimensional vectors x = [1, 2] and y = [3, 4]. Their dot product can be calculated as:</p><p><strong><em>&lt;x, y&gt; = 1 * 3 + 2 * 4 = 3 + 8 = 11</em></strong></p><p>So, the linear kernel similarity score between x and y would be 11.</p><h3><strong>2.</strong> <strong>Polynomial Kernels</strong></h3><p>Polynomial kernels are similar to linear kernels, but they can capture more complex patterns in the data. They map the input data into a higher-dimensional space by transforming it into a polynomial of a specified degree. The transformed data is then passed through a linear activation function to classify the data. Polynomial kernels are often used for non-linear problems, such as polynomial regression or SVMs.</p><p>The equation for the polynomial kernel is given by:</p><p><strong><em>K(x, x’) = (1 + x.x’)^d</em></strong></p><p>where x and x’ are the input features, and d is the degree of the polynomial. A higher degree results in a more complex decision boundary [19].</p><p><strong>For example</strong>, consider a 2-dimensional feature space with input features x = [x1, x2]. The 3rd-degree polynomial kernel function would transform these features into a 6-dimensional space as follows:</p><p><strong><em>K(x, x’) = (1 + x1x’1 + x2x’2)³ = (1 + x1x’1 + x2x’2)(1 + x1x’1 + x2x’2)(1 + x1x’1 + x2x’2) = 1 + 3(x1x’1 + x2x’2) + 6x1x’1x2x’2 + 3(x1²x’1² + x2²x’2²) + x1³x’1³ + x2³x’2³</em></strong></p><p>The new features are a combination of the original features and their powers.</p><h3><strong>3.</strong> <strong>Gaussian Kernels (Radial Basis Function)</strong></h3><p>Gaussian kernels, also known as radial basis function (RBF) kernels, are used for non-linear classification problems. They map the input data into a higher-dimensional space by transforming it into a Gaussian distribution. The transformed data is then passed through a non-linear activation function to classify the data. Gaussian kernels are commonly used for classification problems that involve non-linear boundaries, such as decision trees or neural networks.</p><p>The mathematical formula for a Gaussian kernel is given by:</p><p><strong><em>k(x, x’) = exp(-||x — x’||² / 2σ²)</em></strong></p><p>where x and x’ are the two points being compared, ||x — x’|| is the Euclidean distance between the points, and σ is the standard deviation that determines the shape of the Gaussian curve [20].</p><p><strong>For example</strong>, let’s say we have two data points in a two-dimensional space: x = (1, 2) and x’ = (3, 4). We can calculate the Gaussian kernel between these two points by using the formula above:</p><p><strong><em>k(x, x’) = exp(-||(1, 2) — (3, 4)||² / 2σ²) = exp(-((3–1)² + (4–2)²) / 2σ²) = exp(-5 / 2σ²)</em></strong></p><p>The value of σ determines the shape of the Gaussian curve and the overall value of the kernel. A larger value of σ results in a wider Gaussian curve and a smaller value of the kernel, while a smaller value of σ results in a narrower Gaussian curve and a larger value of the kernel.</p><p>In essence, the Gaussian kernel measures how similar the two points are by computing the normal distribution between them. Points that are closer together will result in a larger kernel value, indicating a higher similarity, while points that are further apart will result in a smaller kernel value, indicating a lower similarity.</p><h3><strong>4.</strong> <strong>Sigmoid Kernels</strong></h3><p>Sigmoid kernels are used for binary classification problems, where the goal is to separate the data into two classes. They map the input data into a higher-dimensional space by transforming it into a sigmoid function. The transformed data is then passed through a sigmoid activation function to classify the data. Sigmoid kernels are often used for logistic regression or SVMs.</p><p>A sigmoid kernel is a type of radial basis function that measures the similarity between two points in a multi-dimensional space by transforming the dot product between the points into a non-linear space. The mathematical formula for a sigmoid kernel is given by:</p><p><strong><em>k(x, x’) = tanh(α * x * x’ + r)</em></strong></p><p>where x and x’ are the two points being compared, α is a scalar that determines the steepness of the sigmoid curve, and r is a bias term that shifts the curve left or right [21, 22].</p><p><strong>For example</strong>, let’s say we have two data points in a two-dimensional space: x = (1, 2) and x’ = (3, 4). We can calculate the sigmoid kernel between these two points by using the formula above:</p><p><strong><em>k(x, x’) = tanh(α * (1, 2) * (3, 4) + r) = tanh(α * (1 * 3 + 2 * 4) + r) = tanh(α * 11 + r)</em></strong></p><p>The value of α determines the steepness of the sigmoid curve, and the value of r determines the position of the curve. A larger value of α results in a steeper sigmoid curve, while a smaller value of α results in a shallower sigmoid curve. A positive value of r shifts the curve to the right, while a negative value of r shifts the curve to the left.</p><p>In essence, the sigmoid kernel measures the similarity between two points by transforming their dot product into a non-linear space. Points that are highly correlated will result in a larger kernel value, indicating a higher similarity, while points that are not correlated will result in a smaller kernel value, indicating a lower similarity.</p><h3><strong>5.</strong> <strong>Laplacian Kernels</strong></h3><p>Laplacian kernels, also known as Laplacian of Gaussian (LoG) kernels, are used in decision trees or neural networks like image processing for edge detection. The Laplacian of the Gaussian function is defined as the second derivative of a Gaussian function. The Laplacian kernel is a 2D filter applied to an image by convolution, where the values in the kernel matrix are the coefficients of the Laplacian of the Gaussian function. The purpose of the Laplacian kernel is to calculate the second derivative of the image intensity, resulting in zero-crossings that correspond to edges in the image. The Laplacian kernel is typically a square matrix with odd dimensions, for example, 3x3 or 5x5, and is symmetrical around the central element.</p><p>Laplacian kernels are used for non-linear classification problems. They map the input data into a higher-dimensional space by transforming it into a Laplacian distribution. The transformed data is then passed through a non-linear activation function to classify the data. The mathematical formula for a Laplacian kernel is given:</p><p><strong><em>k(x, x’) = exp(-||x — x’|| / σ)</em></strong></p><p>where x and x’ are the two points being compared, ||x — x’|| is the Euclidean distance between the points, and σ is a scalar that determines the scale of the kernel [23, 24].</p><p><strong>For example</strong>, let’s say we have two data points in a two-dimensional space: x = (1, 2) and x’ = (3, 4). We can calculate the Laplacian kernel between these two points by using the formula above:</p><p><strong><em>k(x, x’) = exp(-||(1, 2) — (3, 4)|| / σ) = exp(-sqrt((3–1)² + (4–2)²) / σ) = exp(-sqrt(5) / σ)</em></strong></p><p>The value of σ determines the scale of the kernel, with a larger value of σ resulting in a larger kernel value and a smaller value of σ resulting in a smaller kernel value.</p><h3><strong>6.</strong> <strong>Cosine Similarity Kernels</strong></h3><p>Cosine kernels are similarity measures that can be used to compare two vectors in a high-dimensional feature space. Cosine kernels are defined as functions that take two vectors as input and return a scalar value that represents the cosine of the angle between the vectors.</p><p>The cosine similarity between two vectors x and y in a high-dimensional feature space is defined as:</p><p><strong><em>K(x, y) = (x * y) / (||x|| * ||y||)</em></strong></p><p>where (x * y) is the dot product of the vectors x and y, and ||x|| and ||y|| are the Euclidean norms of the vectors.</p><p>The cosine similarity ranges from -1 to 1, where 1 indicates that the vectors are identical (have an angle of 0°), 0 indicates that the vectors are orthogonal (have an angle of 90°), and -1 indicates that the vectors are diametrically opposed (have an angle of 180°) [25–27].</p><p><strong>For example</strong>, let’s consider two vectors x and y in a 3-dimensional feature space, where x = [1, 2, 3] and y = [2, 3, 4]. The cosine similarity between x and y can be computed as follows:</p><p><strong><em>K(x, y) = (x * y) / (||x|| * ||y||) = (1 * 2 + 2 * 3 + 3 * 4) / (sqrt(1² + 2² + 3²) * sqrt(2² + 3² + 4²)) = 0.97</em></strong></p><p>So, in this example, the cosine kernel would compare the two vectors x and y and return a scalar value of 0.97, which indicates a high degree of similarity between the vectors.</p><h3><strong>7.</strong> <strong>Histogram Intersection Kernels</strong></h3><p>Histogram intersection kernels are a type of kernel function that can be used in machine learning to compare histograms, which are representations of the distribution of values in a dataset. The histogram intersection kernel measures the similarity between two histograms by computing the area of overlap between them.</p><p>The histogram intersection kernel is defined as follows:</p><p><strong><em>k(h1, h2) = ∑min(h1(i), h2(i))</em></strong></p><p>where h1 and h2 are the histograms and h1(i) and h2(i) are the values of the histograms at bin i [28, 29].</p><p><strong>For example</strong>, let’s consider two histograms h1 and h2, where h1 represents the distribution of values for feature A and h2 represents the distribution of values for feature B. The histogram intersection kernel would calculate the similarity between h1 and h2 by computing the area of overlap between them. If the two histograms are very similar, then the area of overlap would be large, indicating that the features are highly correlated. On the other hand, if the two histograms are very different, then the area of overlap would be small, indicating that the features are not highly correlated.</p><h3><strong>8.</strong> <strong>Hyperbolic kernels</strong></h3><p>The mathematics of hyperbolic kernels is based on hyperbolic functions. A hyperbolic function is a type of mathematical function that models many important physical and biological processes, including the growth and decay of populations, the spread of diseases, and the diffusion of heat and other physical quantities.</p><p>In the context of machine learning, hyperbolic kernels are often used as a similarity measure between two instances in a feature space. The kernel function takes as input two instances and outputs a scalar value indicating the similarity between them.</p><p>A simple example of a hyperbolic kernel function is the sine hyperbolic kernel, which is defined as:</p><p><strong><em>K(x, y) = sinh(αx^Ty + β)</em></strong></p><p>where α and β are hyperparameters that control the shape of the kernel, and <em>x^Ty</em> is the dot product of the two instances x and y. The dot product measures the similarity between the two instances by counting the number of common features they have. The sine hyperbolic function is then applied to this dot product to produce the final similarity score.</p><p>In general, the choice of the hyperbolic kernel will depend on the specific application and the nature of the data being analyzed. Some hyperbolic kernels may be better suited to certain types of data, while others may perform poorly. It’s important to carefully consider the choice of kernel function when using machine learning techniques and to experiment with different kernel functions to determine which one works best for a given problem [30, 31].</p><h3><strong>When and where each kernel is used?</strong></h3><p>Kernels are commonly used in various machine learning algorithms, particularly in SVM and kernel methods. Here’s a brief overview of when and where each kernel is used:</p><p><strong>Linear kernel:</strong> The linear kernel is the simplest kernel and is used when the data is linearly separable. It is commonly used for simple binary classification problems.</p><p><strong>Polynomial kernel:</strong> The polynomial kernel is used when the data is not linearly separable. It maps the input data into a higher-dimensional feature space where a linear classifier can be applied.</p><p><strong>Radial basis function (RBF) kernel:</strong> The RBF kernel is a commonly used non-linear kernel and is equivalent to the Gaussian kernel. It is used for non-linear classification and regression problems.</p><p><strong>Sigmoid kernel:</strong> The sigmoid kernel is used for binary classification problems and is similar to the logistic regression classifier. It is not commonly used compared to other kernels.</p><p><strong>Laplacian kernel:</strong> The Laplacian kernel is used for non-linear regression problems and is similar to the Gaussian kernel. It is particularly useful for datasets with complex patterns.</p><p><strong>Cosine similarity kernel:</strong> The cosine similarity kernel is used for text classification problems, where the similarity between two documents is calculated based on the cosine of the angle between their vectors in a high-dimensional feature space.</p><p><strong>Histogram intersection kernel:</strong> The histogram intersection kernel is used for image classification problems, where the similarity between two images is calculated based on the intersection of their histograms.</p><p><strong>Hyperbolic kernel:</strong> The hyperbolic kernel is used for non-linear regression problems and is similar to the Gaussian and Laplacian kernels. It is particularly useful for datasets with complex patterns.</p><h3><strong>Implement a Kernel Function in Python</strong></h3><p>As an example, here is a sample code for using the radial basis function (RBF) kernel in Python using the scikit-learn library:</p><pre>import numpy as np<br>from sklearn.svm import SVC<br>from sklearn.datasets import load_iris<br>from sklearn.model_selection import train_test_split<br># Load the iris dataset<br>iris = load_iris ()<br>X = iris.data<br>y = iris.target<br># Split the data into training and testing sets<br>X_train, X_test, y_train, y_test = train_test_split (X, y, test_size=0.2)<br># Train the SVM model with the RBF kernel<br>clf = SVC (kernel=&#39;rbf&#39;)<br>clf.fit (X_train, y_train)<br># Evaluate the model on the test set<br>acc = clf.score (X_test, y_test)<br>print (&quot;Accuracy:&quot;, acc)</pre><h3><strong>Selection of Kernels</strong></h3><p>The choice of kernel depends on the nature of the problem, the type of data, and the computational resources available. It is important to experiment with different kernels to determine the best one for a given problem.</p><p>The choice of the kernel in machine learning can greatly impact the performance and accuracy of a model. For instance:</p><p><strong>1. Linear kernels</strong> are ideal for linear problems.</p><p><strong>2. Polynomial kernels</strong> for non-linear problems.</p><p><strong>3. Gaussian kernels</strong> for non-linear classification problems.</p><p><strong>4. Sigmoid kernels</strong> for binary classification problems.</p><p><strong>5. Laplacian kernels</strong> for non-linear classification problems.</p><p>It is important to understand the strengths and weaknesses of each type of kernel in order to make an informed decision about which one to use for a given problem.</p><h3><strong>Hyperparameter Tuning</strong></h3><p>The performance of a kernel-based algorithm can also be affected by the values of the hyperparameters associated with the kernel. Hyperparameter tuning is the process of finding the optimal values for these hyperparameters to maximize the performance of the algorithm. This is often done using techniques such as cross-validation or grid search.</p><p>There are several approaches to hyperparameter tuning for kernel-based algorithms, including:</p><p><strong>1. Grid search:</strong> In grid search, a set of possible hyperparameter values is defined, and the algorithm is trained and evaluated for each combination of hyperparameter values. The combination with the best performance is then chosen as the final hyperparameter value.</p><p><strong>2. Random search:</strong> In random search, the hyperparameter values are chosen randomly from a predefined distribution, and the algorithm is trained and evaluated. This process is repeated multiple times, and the combination of hyperparameter values with the best performance is chosen as the final hyperparameter values.</p><p><strong>3. Bayesian optimization:</strong> In Bayesian optimization, a probabilistic model is used to predict the performance of the algorithm for different hyperparameter values. The hyperparameter values are then chosen based on the model’s prediction, and the process is repeated until a satisfactory performance is achieved [32, 33].</p><h3><strong>Limitations of kernels</strong></h3><p>Kernels have some limitations, including the choice of kernel, the sensitivity to hyperparameters, and the potential for overfitting. Additionally, kernels can be computationally intensive, especially for large datasets, and can take a long time to train. It’s also important to keep in mind that kernel-based methods may not always perform well when dealing with highly imbalanced or noisy data.</p><p><strong>Overall</strong>, understanding kernels and how to select and tune them is an important aspect of being a data scientist.</p><p>I hope this article gives you an overview of what a data scientist needs to know about kernels and helps you to understand kernels just a little bit better.</p><h3><strong>References</strong></h3><p>[1] <a href="https://scikit-learn.org/stable/modules/svm.html">Cortes, C., &amp; Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273–297</a></p><p>[2] <a href="https://intellipaat.com/blog/tutorial/machine-learning-tutorial/svm-algorithm-in-python/">Support Vector Machine (SVM) Algorithm — Intellipaat</a></p><p>[3] <a href="https://learnopencv.com/support-vector-machines-svm/">Support Vector Machines (SVM) | LearnOpenCV</a></p><p>[4] <a href="https://en.wikipedia.org/wiki/Linear_separability">Baeldung. (2022). Linearly separable data in neural networks</a></p><p>[5] (PDF) <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.121.6481&amp;rep=rep1&amp;type=pdf">The Linear Separability Problem: Some Testing Methods</a></p><p>[6] <a href="https://www.educba.com/what-is-kernel-in-machine-learning/">What is Kernel in Machine Learning? | why do we need | Benefits — EDUCBA</a></p><p>[7] <a href="https://en.wikipedia.org/wiki/Kernel_method">Kernel method — Wikipedia</a></p><p>[8] <a href="https://www.cambridge.org/core/books/kernel-methods-and-machine-learning/4B52092A98E1553A26EB5271D832D29E">Kernel Methods and Machine Learning — cambridge.org</a></p><p>[9] <a href="https://www.educba.com/kernel-methods-in-machine-learning/">Kernel Methods in Machine Learning | Top 7 Types of Kernel Methods — EDUCBA</a></p><p>[10] <a href="https://towardsdatascience.com/the-kernel-trick-c98cdbcaeb3f">The Kernel Trick in Support Vector Classification | by Drew Wilimitis | Towards Data Science</a></p><p>[11] <a href="https://medium.com/@zxr.nju/what-is-the-kernel-trick-why-is-it-important-98a98db0961d">What is the kernel trick? Why is it important? — Medium</a></p><p>[12] <a href="https://www.ling.upenn.edu/courses/ling525/kernel_trick.pdf">The Kernel Trick — ling.upenn.edu</a></p><p>[13] <a href="https://ieeexplore.ieee.org/abstract/document/8292990">Jos Luis Rojo-lvarez; Manel Martnez-Ramn; Jordi Muoz-Mar; Gustau Camps-Valls, ”Support Vector Machine and Kernel Classiﬁcation Algorithms,” in Digital Signal Processing with Kernel Methods , , IEEE, 2018, pp.433–502</a></p><p>[14] <a href="https://en.wikipedia.org/wiki/Reproducing_kernel_Hilbert_space">Reproducing kernel Hilbert space — Wikipedia</a></p><p>[15] <a href="https://arxiv.org/abs/1408.0952">A Primer on Reproducing Kernel Hilbert Spaces — arXiv.org</a></p><p>[16] <a href="https://people.eecs.berkeley.edu/~bartlett/courses/281b-sp08/7.pdf">Reproducing Kernel Hilbert Spaces — University of California, Berkeley</a></p><p>[17] <a href="https://math.stackexchange.com/questions/1040454/reproducing-kernel-hilbert-space-why">Functional analysis — Reproducing kernel Hilbert space, why? — Mathematics Stack Exchange</a></p><p>[18] <a href="https://www.geeksforgeeks.org/creating-linear-kernel-svm-in-python/%C2%A0">https://www.geeksforgeeks.org/creating-linear-kernel-svm-in-python</a></p><p>[19] Baeldung. (2022). Polynomial kernel in neural networks. <a href="https://www.baeldung.com/cs/nn-polynomial-kernel">https://www.baeldung.com/cs/nn-polynomial-kernel</a> <a href="https://en.wikipedia.org/wiki/Polynomial_kernel">1</a></p><p>[20] Natsume, Y. (2022). Gaussian process kernels. More than just the radial basis function. <a href="https://towardsdatascience.com/gaussian-process-kernels-96bafb4dd63e">https://towardsdatascience.com/gaussian-process-kernels-96bafb4dd63e</a> <a href="https://en.wikipedia.org/wiki/Gaussian_filter">1</a></p><p>[21] sklearn.metrics.pairwise.sigmoid_kernel — scikit-learn</p><p>[22] Support Vector Machine (SVM) and Kernels Trick — Medium</p><p>[23] <a href="https://en.wikipedia.org/wiki/Discrete%20Laplace%20operator">https://en.wikipedia.org/wiki/Discrete Laplace operator</a></p><p>[24] <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.laplacian_kernel.html">sklearn.metrics.pairwise.laplacian_kernel — scikit-learn 1.2.2 documentation</a></p><p>[25] <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html">sklearn.metrics.pairwise.cosine_similarity — scikit-learn 1.2.2 documentation</a></p><p>[26] <a href="https://math.stackexchange.com/questions/1543393/proving-cosine-is-valid-kernel">Matrices — Proving cosine is valid kernel — Mathematics Stack Exchange</a></p><p>[27] <a href="https://stackoverflow.com/questions/56316903/cosine-similarity-between-0-and-1">Python — Cosine similarity between 0 and 1 — Stack Overflow</a></p><p>[28] <a href="https://ieeexplore.ieee.org/document/1247294">Histogram intersection kernel for image classification | IEEE Conference Publication | IEEE Xplore</a></p><p>[29] <a href="https://www.researchgate.net/publication/4044656_Histogram_intersection_kernel_for_image_classification">(PDF) Histogram intersection kernel for image classification (researchgate.net)</a></p><p>[30] <a href="https://ieeexplore.ieee.org/document/9709915">Kernel Methods in Hyperbolic Spaces | IEEE Conference Publication | IEEE Xplore</a></p><p>[31] <a href="https://datascience.stackexchange.com/questions/10479/on-the-properties-of-hyperbolic-tangent-kernel">Machine learning — On the properties of Hyperbolic Tangent Kernel — Data Science Stack Exchange</a></p><p>[32] <a href="https://iopscience.iop.org/article/10.1088/2632-2153/abee59">Efficient hyperparameter tuning for kernel ridge regression with Bayesian optimization — IOPscience</a></p><p>[33] <a href="https://scikit-learn.org/stable/modules/grid_search.html">Tuning the hyper-parameters of an estimator — scikit-learn 1.2.2 documentation</a></p><h3><strong>Author: Ali Farahmandfar</strong></h3><h4>Contact: <a href="https://github.com/alifrmf">Github</a></h4><p><a href="https://github.com/alifrmf">alifrmf - Overview</a></p><h4><a href="https://mlearning.substack.com/about">BECOME a WRITER at MLearning.ai</a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4066f02d1f26" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do I choose a machine learning algorithm for my application?]]></title>
            <link>https://melanee-melanee.medium.com/how-do-i-choose-a-machine-learning-algorithm-for-my-application-1699967bb31a?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/1699967bb31a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Mon, 13 Mar 2023 07:32:24 GMT</pubDate>
            <atom:updated>2023-03-28T08:55:42.153Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="How do I choose a machine learning algorithm for my application?" src="https://cdn-images-1.medium.com/max/1024/0*UywdXXgZTliac-Ml" /><figcaption>Photo by <a href="https://unsplash.com/@askkell?utm_source=medium&amp;utm_medium=referral">Andy Kelly</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Choosing a machine learning (ML) or deep learning (DL) algorithm for application is one of the major issues for artificial intelligence (AI) engineers and also data scientists. Here I want to clarify this issue.</p><p>In this regard, choosing an appropriate algorithm is based on different factors, such as the type of data you have, the problem you are struggling to solve, and the available resources. Here are some steps which help you to address this issue:</p><h4>Define the problem:</h4><p>Identify the problem you are trying to solve and the type of data you have. Is it a classification problem or a regression problem? Do you have labeled or unlabeled data?</p><h4>Determine the available resources:</h4><p>Consider the computational resources, time, and configurations available to you. Some algorithms, especially, deep learning algorithms may require a lot of computational power, for instance, powerful CPU, GPU and RAM.</p><h4>Explore algorithms:</h4><p>Research and explore different algorithms that are desired for your problem. You can start with simpler algorithms such as decision trees, <a href="https://en.wikipedia.org/wiki/Naive_Bayes_classifier">Naive Bayes</a>, and logistic regression, and steadily move to more complex ones like neural networks and support vector machines (SVM).</p><h4>Experiment and evaluate:</h4><p>Implement the algorithms you have selected and evaluate their performance on your data. Use metrics such as accuracy, precision, recall, and <a href="https://towardsdatascience.com/the-f1-score-bec2bbc38aa6">F1 score</a> to evaluate the model’s performance.</p><h4>Iterate:</h4><p>Based on the evaluation results, iterate and fine-tune the model to improve its performance.</p><h4>Deploy the model:</h4><p>Once you have selected and fine-tuned the algorithm, deploy it in your application.</p><p>In table 1 and table 2, you can see the applications of some supervised and unsupervised ML algorithms and also DL algorithms at a glance.</p><figure><img alt="ML algorithms and their application" src="https://cdn-images-1.medium.com/max/1024/1*x1kbjVALDwH28NqOP4AulQ.png" /><figcaption>Table 1. ML algorithms and their application [table by author]</figcaption></figure><figure><img alt="deep learning algorithms and their application" src="https://cdn-images-1.medium.com/max/1024/1*LIbzp4HA5hfpMwveOY2wrA.png" /><figcaption>Table 2. DL algorithms and their application [table by author]</figcaption></figure><p>In conclusion, choosing the right algorithm can dramatically affect the performance of your application, so spend your time to test with different options.</p><p>If this article has been able to add to your knowledge, please share it with your colleagues and link it to your stories.</p><h3>References for more studies:</h3><p>[1] <a href="https://labelyourdata.com/articles/how-to-choose-a-machine-learning-algorithm">https://labelyourdata.com/articles/how-to-choose-a-machine-learning-algorithm</a></p><p>[2] <a href="https://github.com/Melanee-Melanee/scikit-learn">https://github.com/Melanee-Melanee/scikit-learn</a></p><h3>Author:</h3><p>Melanee</p><p>GitHub: <a href="https://github.com/Melanee-Melanee">https://github.com/Melanee-Melanee</a></p><p><a href="https://github.com/Melanee-Melanee">Melanee-Melanee - Overview</a></p><h4><a href="https://mlearning.substack.com/about"><strong>BECOME a WRITER at MLearning.ai</strong></a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1699967bb31a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ChatGPT Replies About Adaptive Learning]]></title>
            <link>https://levelup.gitconnected.com/chatgpt-replies-about-adaptive-learning-5ae71807a43a?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ae71807a43a</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[adaptive-learning]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[chatgpt3]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Mon, 23 Jan 2023 20:07:10 GMT</pubDate>
            <atom:updated>2023-02-02T18:07:35.510Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Adaptive Learning" src="https://cdn-images-1.medium.com/max/1024/1*C0idqJN-tHu-mT7AyScuOg.jpeg" /><figcaption>Photo taken from<a href="https://urdutechnology.com/entertainment/chatgpt/"> Urdutechnology</a></figcaption></figure><blockquote>Most of this content is generated by <a href="https://openai.com/blog/chatgpt/">ChatGPT</a>.</blockquote><blockquote>It uses the artificial intelligence of <a href="https://en.wikipedia.org/wiki/OpenAI">OpenAI</a>.</blockquote><p><strong>Melanee: What is adaptive learning?</strong></p><p><strong>ChatGPT</strong>: Adaptive learning is a type of educational technology that uses artificial intelligence (AI) algorithms to personalize the learning experience for each student. It adapts the teaching methods, content, and pace based on the student’s performance, preferences, and learning style. Adaptive learning systems use data from the student’s interactions with the system, such as their answers to questions, their performance on assessments, and their learning patterns, to adjust the instruction and provide personalized feedback.</p><p>Adaptive learning systems can be used in a variety of educational settings, such as K-12 schools, higher education, and corporate training. They can be used to deliver instruction in traditional classroom settings, online, or through mobile devices. Adaptive learning systems can also be integrated with other educational technologies, such as virtual reality and gamification, to create more engaging and interactive learning experiences.</p><p>The goal of adaptive learning is to improve student outcomes by providing personalized instruction that addresses the unique needs and strengths of each student. By providing students with tailored instruction, adaptive learning systems aim to help students learn more efficiently and effectively, and to close the achievement gap among students.</p><p><strong>Melanee: Which AI algorithms have been used in Adaptive learning?</strong></p><p><strong>ChatGPT: </strong>There are several AI algorithms that are commonly used in adaptive learning, including:</p><ol><li>Reinforcement Learning: This type of algorithm learns by trial and error, where the model receives feedback in the form of rewards or penalties for its actions. Reinforcement learning is used in adaptive learning to adjust the model’s parameters based on the feedback it receives.</li><li>Neural Networks: Neural networks are a type of machine learning algorithm that are designed to mimic the structure and function of the human brain. They are used in adaptive learning to adjust the model’s parameters based on the feedback it receives.</li><li>Bayesian Methods: Bayesian methods are a family of statistical models that are used to update the model’s parameters based on new data. These methods are used in adaptive learning to update the model’s parameters based on feedback it receives.</li><li>Genetic Algorithm: Genetic algorithms are inspired by the process of natural selection and are used to find the optimal set of model’s parameters by simulating the process of natural evolution.</li><li>Gradient Descent: Gradient descent is an optimization algorithm that is used to find the set of model’s parameters that minimize the error.</li><li>Particle Swarm Optimization (PSO): PSO is an optimization algorithm that is inspired by the behavior of birds flocking or fish schooling. It is used to find the optimal set of model’s parameters.</li></ol><p>It’s important to note that the choice of algorithm depends on the use case, the data availability and the computational resources. And also, Adaptive learning is a continuous process and the models need to be continuously updated based on the new data and feedback.</p><p><strong>Melanee: what are related keywords about adaptive learning?</strong></p><p><strong>ChatGPT: </strong>Some related keywords about adaptive learning include:</p><ol><li>Personalization: Adaptive learning systems use a variety of techniques to personalize the learning experience for each individual student, tailoring the content and pace to their unique needs and abilities.</li><li>Machine learning: Adaptive learning relies heavily on machine learning algorithms to analyze student data and adapt the learning experience in real-time.</li><li>Intelligent Tutoring Systems: Adaptive learning systems often incorporate intelligent tutoring systems which use AI to provide customized feedback and guidance to students.</li><li>Learning analytics: Adaptive learning systems use learning analytics to track student progress and make data-driven decisions about how to adapt the learning experience.</li><li>Educational data mining: Adaptive learning systems use educational data mining to extract useful information from student data in order to personalize the learning experience.</li><li>Personalized learning: Adaptive learning systems provide a personalized learning experience by tailoring the content and pace of instruction to the student’s needs, preferences and abilities.</li><li>Intelligent Adaptation: Adaptive learning systems use AI techniques to adapt the learning experience to the student’s level of understanding, learning style and preferences.</li><li>Self-Regulated Learning: Adaptive learning systems support students in becoming more self-regulated learners by providing them with personalized feedback and guidance.</li></ol><p><strong>Melanee: how user data is feeded in adaptive learning to create an ai model?</strong></p><p><strong>ChatGPT: </strong>In adaptive learning, user data is typically fed into the AI model in a process called “training”. The training process involves feeding the model large amounts of data, known as “training data”, and adjusting the model’s parameters in order to improve its performance. There are several ways in which user data can be fed into the adaptive learning process:</p><ol><li>Online learning: User data is fed into the AI model in real-time as the model is being used. This allows the model to continuously learn and improve its performance over time.</li><li>Batch learning: User data is collected over time and periodically fed into the model in batches. This allows the model to learn from a large amount of data at once.</li><li>Active learning: User data is selectively chosen and fed into the model in order to maximize the model’s learning potential. This can be done by identifying the data that is most informative or relevant to the model’s task.</li><li>Transfer learning: User data from different users or contexts can be used to transfer knowledge from one domain to another.</li><li>Data augmentation: User data can be used to augment the AI model’s training data, which can help the model generalize better and improve its performance on unseen data.</li></ol><p>It is important to note that the user data should be preprocessed, cleaned and pre-analyzed before feeding into the model. Also, the data should be diverse, representative and unbiased to avoid any problem of overfitting or poor generalization of the mod<em> (sentence was not generated completely!).</em></p><p><strong>Melanee: can you explain about real time models?</strong></p><p><strong>ChatGPT: </strong>In the context of AI and machine learning, real-time models refer to models that are able to process and make predictions or decisions based on incoming data in real-time, that is, with little to no delay. These models are designed to operate in dynamic environments, where the data is constantly changing, and the model must be able to adapt and make predictions or decisions on the fly.</p><p>Real-time models are used in various applications, such as:</p><ol><li>Robotics: Real-time models are used to control robots and make decisions based on sensor data in real-time.</li><li>Video and Image processing: Real-time models are used to process video and image data in real-time, for tasks such as object detection, facial recognition, and image segmentation.</li><li>Automotive: Real-time models are used in advanced driver assistance systems (ADAS) and autonomous vehicles to make decisions based on sensor data in real-time.</li><li>Industrial Automation: Real-time models are used in industrial automation to control machinery and make decisions based on sensor data in real-time</li><li>Online Personalization: Real-time models are used to personalize the online experience for users by analyzing their behavior and making predictions in real-time.</li></ol><p>Real-time models have to be able to process large amount of data in a short period of time, and should be optimized for speed and resource efficiency. To achieve this, they are often deployed on specialized hardware such as Graphics Processing Units (GPUs) or Field-Programmable Gate Arrays (FPGAs) and use specialized libraries such as TensorRT or OpenVINO.</p><p><strong>Melanee: Can you show me a python code related to adaptive learning?</strong></p><p><strong>ChatGPT:</strong> Sure, here is an example of a Python code for implementing an adaptive learning algorithm using the Keras library:</p><pre>from keras.layers import Dense<br>from keras.models import Sequential<br>from sklearn.datasets import make_classification<br>from sklearn.metrics import accuracy_score<br># Generate dataset<br>X, y = make_classification(n_samples=1000, n_features=4, n_informative=2, n_redundant=0, random_state=0, shuffle=False)<br># Split the dataset into training and test sets<br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)<br># Create the model<br>model = Sequential()<br>model.add(Dense(32, input_dim=4, activation=’relu’))<br>model.add(Dense(1, activation=’sigmoid’))<br># Compile the model<br>model.compile(optimizer=’adam’, loss=’binary_crossentropy’, metrics=[‘accuracy’])<br># Train the model on the training data<br>model.fit(X_train, y_train, epochs=10, batch_size=32)<br># Make predictions on the test data<br>y_pred = model.predict_classes(X_test)<br># Calculate the accuracy<br>acc = accuracy_score(y_test, y_pred)<br>print(“Accuracy:”, acc)<br># Update the model with new data<br>X_update, y_update = make_classification(n_samples=200, n_features=4, n_informative=2, n_redundant=0, random_state=0, shuffle=False)<br># Train the model on the updated data<br>model.fit(X_update, y_update, epochs=10, batch_size=32)<br># Make predictions on the updated data<br>y_pred = model.predict_classes(X_test)<br># Calculate the updated accuracy<br>acc = accuracy_score(y_test, y_pred)<br>print(&quot;Updated accuracy:&quot;, acc)</pre><blockquote>Check all of codes on my GitHub account:</blockquote><blockquote><a href="https://github.com/Melanee-Melanee/ChatGPT-Melanee">https://github.com/Melanee-Melanee/ChatGPT-Melanee</a></blockquote><p>If you find this article informative, please share and clap it.</p><p>Thanks in advance.</p><p>23 Jan 2023</p><p>Authors: <a href="https://www.kaggle.com/melaneemelanee">Melanee</a> and ChatGPT</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ae71807a43a" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/chatgpt-replies-about-adaptive-learning-5ae71807a43a">ChatGPT Replies About Adaptive Learning</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DDoS attacks via new Python packages]]></title>
            <link>https://melanee-melanee.medium.com/ddos-attacks-via-new-python-packages-5df946e42d1f?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/5df946e42d1f</guid>
            <category><![CDATA[pypi]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[malicious-code]]></category>
            <category><![CDATA[ddos-attack]]></category>
            <category><![CDATA[python-packages]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Mon, 12 Sep 2022 08:46:54 GMT</pubDate>
            <atom:updated>2022-09-12T13:27:18.905Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="malicious python packages" src="https://cdn-images-1.medium.com/max/647/1*LZeP-PR0QqnO-npeaOKItg.jpeg" /><figcaption>Photo by Zhukov Oleg</figcaption></figure><p>This weekend, researchers at <a href="https://checkmarx.com/"><strong>Checkmarx</strong></a> discovered that a user named “devfather777” published 12 python packages on <a href="https://pypi.org/"><strong>PyPi</strong></a> that used a name similar to other popular packages to trick software developers into using the malicious versions instead [1].</p><p>The complete list of uploaded malicious <strong>PyPi</strong> packages are:</p><pre>Gesnim<br>Kears<br>TensorFolw<br>Seabron<br>tqmd<br>lxlm<br>mokc<br>ipaddres<br>ipadress<br>falsk<br>douctils<br>inda</pre><blockquote>A dozen malicious Python packages were uploaded to the PyPi repository this weekend in a typosquatting attack that performs DDoS attacks on a Counter-Strike 1.6 server [2].</blockquote><p>Because software developers usually fetch these packages via the terminal, it’s easy to type their name with a letter in the wrong order. Since the download and build continue as expected, the victim doesn’t realize the mistake and infects their device [1].</p><p><a href="https://www.malwarebytes.com/ddos"><strong>DDoS</strong></a> (Distributed Denial of Service) involves sending large amounts of traffic from multiple sources to a service or website, intending to overwhelm it. A huge influx of traffic all at once can tie up all the site’s resources and thereby deny access to legitimate users.</p><p>If you use any of the 12 packages mentioned above and think you made a typo this weekend, go over your projects and double-check that you are using the right software. Unfortunately, no <strong>PyPI</strong> package comes with security guarantees. Users are responsible for double-checking names, or any other details such as release histories, submission details, homepage links, and download numbers [3].</p><p>Hopefully, I (Melanee) have checked the <strong>PyPi</strong> repository and that detrimental packages have been removed from there, please share this story with your friends to make them aware to uninstall these packages.</p><h3>Writer: Melanee</h3><p>Contact Melanee on <a href="https://github.com/Melanee-Melanee">GitHub</a>:</p><p><a href="https://github.com/Melanee-Melanee">Melanee-Melanee - Overview</a></p><h3>References:</h3><p>[1] <a href="https://www.bleepingcomputer.com/news/security/malicious-pypi-packages-aim-ddos-attacks-at-counter-strike-servers/">https://www.bleepingcomputer.com/news/security/malicious-pypi-packages-aim-ddos-attacks-at-counter-strike-servers/</a></p><p>[2] <a href="https://www.reddit.com/r/programming/comments/wpk3uj/a_dozen_malicious_python_packages_were_uploaded/">https://www.reddit.com/r/programming/comments/wpk3uj/a_dozen_malicious_python_packages_were_uploaded/</a></p><p>[3] <a href="https://devm.io/python/malicious-pypi-packages">https://devm.io/python/malicious-pypi-packages</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5df946e42d1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IP version 6 at a glance]]></title>
            <link>https://melanee-melanee.medium.com/ip-version-6-at-a-glance-36a055ba0dc6?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/36a055ba0dc6</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[ip]]></category>
            <category><![CDATA[ipv6]]></category>
            <category><![CDATA[computer-network]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Mon, 20 Jun 2022 06:29:12 GMT</pubDate>
            <atom:updated>2022-06-21T07:44:27.170Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="WAN networking" src="https://cdn-images-1.medium.com/max/612/1*kb9hrQCSM1g9scmKmKnBtQ.jpeg" /><figcaption><a href="https://www.istockphoto.com/photo/people-network-and-global-earth-connection-in-innovative-perception-gm1303551046-394975133">https://www.istockphoto.com/photo/people-network-and-global-earth-connection-in-innovative-perception-gm1303551046-394975133</a></figcaption></figure><h4><strong>Abstract:</strong></h4><p>For sure, you know the IP address, but if you do not know, it is a 32-bit address that all devices connected to the Internet, such as smartphones, laptops and routers, must have manually or dynamically. They got it, but in this article we do not want to talk about IP version 4, we want to talk about version 6 with some kinds of addressing called EUI-64 … Let’s get to know each other.</p><h4><strong>Introduction</strong>:</h4><p>As you know, we are currently using IP version 4 to connect to the Internet and network. But due to its limited number, it can IP 2 ^ 32 devices. While with the advent of technologies like the Internet of Things (IOT), this number of IP is no longer our answer, now is the time to use IP version 6.</p><h4><strong>IP version 6:</strong></h4><p>To address this, we must first get acquainted with the IP structure of version 6. IP version 6 is a 128-bit hexadecimal address that can be 2¹²⁸ IP the number. In IP version 6 structuring and writing, each 16-bit is separated by the following, each called a 16-bit <a href="https://www.techtarget.com/whatis/definition/hexadecimal">hexadecimal </a>component.</p><figure><img alt="IP components version 6" src="https://cdn-images-1.medium.com/max/1024/1*hBuDdUrxWz2kbaG0c2135Q.jpeg" /><figcaption>Global Address: The address provided by your<a href="https://www.techtarget.com/whatis/definition/ISP-Internet-service-provider"> ISP</a></figcaption></figure><p>Global Address<strong>: </strong>The address provided by your ISP</p><p>Subnet: It is the part that separates the different parts of the network and it is determined by the network administrator.</p><p>HOST ID: This part of the 64-bit address is also specified by the network administrator for the clients they need to share which must be equal.</p><figure><img alt="Server room connections" src="https://cdn-images-1.medium.com/max/1024/0*la_p96VOOU4XuEK3" /><figcaption>Photo by <a href="https://unsplash.com/@tvick?utm_source=medium&amp;utm_medium=referral">Taylor Vick</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4><strong>IP giving by EUI-64 method:</strong></h4><p>EUI-64 : Well, now we come to the EUI-64 method, which uses a physical address or a<a href="https://en.wikipedia.org/wiki/MAC_address"> MAC address</a> of the clients on the network and a 16-bit address to address the clients that need to be shared on the network with IP version 6. Slowly.</p><p>Suppose a MAC address is one of the clients in the network as follows: 00E0.F988.A23B</p><p>And that 16-bit address: FFFE</p><p>Global Address+SUBNET: The address that your ISP has assigned to the network:2001:DB8:6783:A</p><p>step 1: Computing: Well, first there is the ISP and SUBNET addresses:2001:DB8:6783:A</p><p>Step 2: Add 16 bits of the client MAC address to it:2001:DB8:6783:A:00E0</p><p>Step 3: For the next 16 bits, we put the first 8 bits of the MAC address and the next 8 bits of that 16-bit address:2001:DB8:6783:A:00E0:F9FF</p><p>Step 4: For the next 16 bits, place the first 8 bits between the other 16-bit address and the next 8 bits of that MAC address:2001:DB8:6783:A:00E0:F9FF:FE88</p><p>Step 5: Add the remaining 16 bits of the MAC address to the address to complete our 128-bit address:2001:DB8:6783:A:00E0:F9FF:FE88:A23B</p><p>In the last step: we change the third bit of the second number of the MAC address That is, the number 0 in the MAC address changes to 2 And finally our address is the opposite:2001:DB8:6783:A:02E0:F9FF:FE88:A23B</p><p>Well, as I said, it was the number 0, whose language is hexadecimal, and if we convert it to binary, it becomes 0000, which if we change the third bit, it means 1, it becomes 0010, which in the language of hexadecimal means the number 2.</p><figure><img alt="One switch connections" src="https://cdn-images-1.medium.com/max/1024/0*aBK8YywqL9KENOsQ" /><figcaption>Photo by <a href="https://unsplash.com/@idgeek?utm_source=medium&amp;utm_medium=referral">Samuel Ramos</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>After you do this for all the clients you want to be on the network and give them version 6 in <a href="https://www.techtarget.com/searchnetworking/definition/TCP-IP">TCP / IP</a> settings, they will be connected.</p><p>Well, we have reached the end of this article. Thank you for being with me until the end of this article, if you find this article please follow our Medium account and share the link of this article with your friends.</p><p><strong>WRITER </strong>: <strong>Alireza Nasri</strong></p><p><strong>Contact link</strong> : <a href="https://medium.com/@alirezanasri.p8888"><strong>Medium</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36a055ba0dc6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to solve the Apriori algorithm in a simple way from scratch?]]></title>
            <link>https://levelup.gitconnected.com/how-to-solve-the-apriori-algorithm-in-a-simple-way-from-scratch-9540cfc5c11a?source=rss-8273fce06f66------2</link>
            <guid isPermaLink="false">https://medium.com/p/9540cfc5c11a</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python-code]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[apriori-algorithm]]></category>
            <dc:creator><![CDATA[Melanee Group]]></dc:creator>
            <pubDate>Sun, 19 Jun 2022 11:52:15 GMT</pubDate>
            <atom:updated>2022-06-19T11:52:15.196Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jAv5qpTW930FA629StRcUA.jpeg" /><figcaption>Image <a href="https://www.journaldev.com/44792/apriori-algorithm">source</a></figcaption></figure><blockquote>Note: All the contents of the images, including tables and calculations and codes have been investigated by me and there is no need to refer any references for them.</blockquote><h3>Introduction</h3><p>There are several methods for <a href="https://www.javatpoint.com/machine-learning">machine learning</a> such as <a href="https://deepai.org/machine-learning-glossary-and-terms/association-learning">association</a>, <a href="https://medium.com/analytics-vidhya/what-is-correlation-4fe0c6fbed47">correlation</a>, <a href="https://towardsdatascience.com/machine-learning-classifiers-a5cc4e1b0623">classification </a>&amp; <a href="https://machinelearningmastery.com/clustering-algorithms-with-python/">clustering</a>, this tutorial primarily focuses on learning using association rules. By association rules, we identify the set of items or attributes that occur together in a table[1].</p><h3>Association Rule Learning</h3><p>The <a href="https://deepai.org/machine-learning-glossary-and-terms/association-learning">association</a> rule learning is one of the very important concepts of <a href="https://www.javatpoint.com/machine-learning">machine learning</a>, and it is employed in <strong>Market Basket analysis, Web usage mining, continuous production, etc.</strong> Here market basket analysis is a technique used by the various big retailer to discover the associations between items. We can understand it by taking an example of a supermarket, as in a supermarket, all products that are purchased together are put together[2].</p><p>Association rule learning can be divided into three types of algorithms[2]:</p><ol><li><strong>Apriori</strong></li><li><strong>Eclat</strong></li><li><strong>F-P Growth Algorithm</strong></li></ol><h3>Introduction to APRIORI</h3><p>Apriori is an <a href="https://www.britannica.com/science/computer-science/Algorithms-and-complexity">algorithm </a>used for Association Rule learning. It searches for a series of frequent sets of items in the datasets. It builds on associations and correlations between the itemsets. It is the algorithm behind “You may also like” that you commonly saw in recommendation platforms[3].</p><figure><img alt="Apriori Algorithem" src="https://cdn-images-1.medium.com/max/700/1*ivVrrcaltUDoVzWdw0748A.jpeg" /><figcaption><strong>Figure1. Apriori [3]</strong></figcaption></figure><h4>What is an Apriori algorithm?</h4><p>Apriori <a href="https://www.britannica.com/science/computer-science/Algorithms-and-complexity">algorithm </a>assumes that any subset of a frequent itemset must be frequent. Say, a transaction containing {milk, eggs, bread} also contains {eggs, bread}. So, according to the principle of Apriori, if {milk, eggs, bread} is frequent, then {eggs, bread} must also be frequent [4].</p><h3>How Does the Apriori Algorithm Work?</h3><p>In order to select the interesting rules out of multiple possible rules from this small business scenario, we will be using the following measures[4]:</p><ul><li><strong>Support</strong></li><li><strong>Confidence</strong></li><li><strong>Lift</strong></li><li><strong>Conviction</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*xMlBaEFymfCS7JtFDg3svQ.png" /><figcaption><strong>Figure2. Apriori Algorithm Work [7]</strong></figcaption></figure><h3>Support</h3><p>Support of item <em>x</em> is nothing but the ratio of the number of transactions in which item <em>x</em> appears to the total number of transactions.</p><h3>Confidence</h3><p>Confidence (<em>x</em> =&gt; <em>y</em>) signifies the likelihood of the item <em>y</em> being purchased when item <em>x</em> is purchased. This method takes into account the popularity of item <em>x</em>.</p><h3>Lift</h3><p>Lift (<em>x</em> =&gt; <em>y</em>) is nothing but the ‘interestingness’ or the likelihood of the item <em>y</em> being purchased when item <em>x</em> is sold. Unlike confidence (<em>x</em> =&gt; <em>y</em>), this method takes into account the popularity of the item <em>y</em>.</p><ul><li><strong>Lift (<em>x </em>=&gt; <em>y</em>)</strong> = 1 means that there is no correlation within the itemset.</li><li><strong>Lift (<em>x</em> =&gt; <em>y</em>)</strong> &gt; 1 means that there is a positive <a href="https://medium.com/analytics-vidhya/what-is-correlation-4fe0c6fbed47">correlation </a>within the itemset, i.e., products in the itemset, <em>x</em> and <em>y</em>, are more likely to be bought together.</li><li><strong>Lift (<em>x</em> =&gt; <em>y</em>)</strong> &lt; 1 means that there is a negative <a href="https://medium.com/analytics-vidhya/what-is-correlation-4fe0c6fbed47">correlation </a>within the itemset, i.e., products in itemset, <em>x</em> and <em>y</em>, are unlikely to be bought together.</li></ul><h3>Conviction</h3><p>Conviction of a rule can be defined as follows:</p><figure><img alt="The formula of Conviction" src="https://cdn-images-1.medium.com/max/109/0*BirEb6MzYN2WP8-i.png" /><figcaption><strong>Figure 3. The formula of Conviction [4]</strong></figcaption></figure><p>Its value range is [0, +∞].</p><ul><li><strong>Conv(<em>x</em> =&gt; <em>y</em>) = 1</strong> means that <em>x</em> has no relation with <em>y</em>.</li><li>Greater the conviction higher the interest in the rule.</li></ul><figure><img alt="Formulae for support, confidence and lift for the association rule X ⟹ Y" src="https://cdn-images-1.medium.com/max/800/1*3eQ7CesIRbMiZ2JePUdpjA.png" /><figcaption><strong>Figure 4. Formulae for support, confidence and lift for the association rule X ⟹ Y [5]</strong></figcaption></figure><h3>Now, we want to solve a problem of the Apriori algorithm in a simple way:</h3><h4>Part(a): Apply the Apriori <a href="https://www.britannica.com/science/computer-science/Algorithms-and-complexity">algorithm </a>to the following data set:</h4><figure><img alt="The set of items including milk, bread, egg, cookie, coffee and juice" src="https://cdn-images-1.medium.com/max/873/1*2_wzA0N-ytk6ya9R9nNi3A.jpeg" /><figcaption><strong>Figure 5. The set of items including milk, bread, egg, cookie, coffee and juice</strong></figcaption></figure><h4>Step-1:</h4><p>In the first step, we index the data and then calculate the support for each one, if support was less than the minimum value we eliminate that from the table.</p><figure><img alt="index the data" src="https://cdn-images-1.medium.com/max/919/1*7sZy0tdalHYz3AXYDqfMfA.png" /><figcaption><strong>Figure 6. Index the data</strong></figcaption></figure><h4>Step-2:</h4><p>Calculate the support for each one</p><figure><img alt="Calculate the support for each one" src="https://cdn-images-1.medium.com/max/831/1*xiO6tZqDLeHOc1ED5XyMTg.png" /><figcaption><strong>Figure 7. Calculate the support for each one</strong></figcaption></figure><h4>Step-3:</h4><p>Continue to calculate the support and select the best answer</p><figure><img alt="Continue to calculate the support and select the best answer" src="https://cdn-images-1.medium.com/max/908/1*BhKcwrfIPAlvsVIpXUYiJQ.jpeg" /><figcaption><strong>Figure 8. Continue to calculate the support and select the best answer</strong></figcaption></figure><h4>Part(b): Show two rules that have a confidence of 70% or greater for an itemset containing three items from part a.</h4><h4>Step-1:</h4><p>Calculate the confidence and follow the rules of question in part(b)</p><figure><img alt="Calculate the confidence" src="https://cdn-images-1.medium.com/max/787/1*hr34WAD1r3QMyEdJt31Mdg.jpeg" /><figcaption><strong>Figure 9. Calculate the confidence</strong></figcaption></figure><h4>Step-2:</h4><p>In addition to the above rules, the following can also be considered, but in the question only two rules are required for calculation.</p><figure><img alt="Rules that have a confidence of 70% or greater" src="https://cdn-images-1.medium.com/max/787/1*spVbFUgjYhFu_aS9UHoSRw.png" /><figcaption><strong>Figure 10. Rules that have a confidence of 70% or greater</strong></figcaption></figure><h3>Hands-on: Apriori Algorithm in Python- Market Basket Analysis</h3><h4>Problem Statement:</h4><p>For the implementation of the Apriori algorithm, we are using data collected from a SuperMarket, where each row indicates all the items purchased in a particular transaction.</p><blockquote>The manager of a retail store is trying to find out an association rule between items, to figure out which items are more often bought together so that he can keep the items together in order to increase sales.<br>The dataset has 7,500 entries. Drive link to download <a href="https://drive.google.com/file/d/1nGzgX2cq8R4AT-9cMq_puzB1XgPnb6Pf/view"><strong>dataset</strong></a>[4][6].</blockquote><h3>Environment Setup:</h3><p>Before we move forward, we need to install the ‘apyori’ package first on command prompt.</p><figure><img alt="Environment Setup" src="https://cdn-images-1.medium.com/max/382/1*BuwVTV6dcOW1PuzKiriuJA.jpeg" /><figcaption><strong>Figure 11. Environment Setup</strong></figcaption></figure><h3>Market Basket Analysis Implementation within Python</h3><p>With the help of the <strong>apyori</strong> package, we will be implementing the Apriori algorithm in order to help the manager in market basket analysis [4].</p><figure><img alt="which items to keep together?" src="https://cdn-images-1.medium.com/max/1024/1*uUTyXbR7RgRsfoJ4LBmP1A.png" /><figcaption><strong>Figure 12. Which items to keep together? [4]</strong></figcaption></figure><h4>Step-1: We import the necessary libraries required for the implementation</h4><pre>import numpy as np<br>import matplotlib.pyplot as plt<br>import pandas as pd</pre><h4>Step-2: Load the dataset</h4><p>Now we have to proceed by reading the dataset we have, that is in a csv format. We do that using pandas module’s read_csv function [6].</p><pre>dataset = pd.read_csv(&quot;Market_Basket_Optimisation.csv&quot;)</pre><h4>Step-3: Take a glance at the records</h4><pre>dataset</pre><figure><img alt="ave a glance at the records" src="https://cdn-images-1.medium.com/max/963/1*mrpqFNkKU-ZWYI34Q1NXtw.png" /><figcaption><strong>Figure 13. Take a glance at the records</strong></figcaption></figure><h4>Step-4: Look at the shape</h4><pre>dataset.shape</pre><figure><img alt="dataset shape" src="https://cdn-images-1.medium.com/max/294/1*swSlLVk1x8N0uvadS8IBJg.png" /><figcaption><strong>Figure 14. Dataset shape</strong></figcaption></figure><h4>Step-5: Convert Pandas DataFrame into a list of lists</h4><pre>transactions = []<br>for i in range(0, 7501):<br> transactions.append([str(dataset.values[i,j]) for j in range(0,20)])</pre><h4>Step-6: Build the Apriori model</h4><p>We import the apriori function from the apyori module. We store the resulting output from apriori function in the ‘rules’ variable. <br> To the apriori function, we pass 6 parameters:</p><ol><li>The transactions List as the main inputs</li><li>Minimum support, which we set as 0.003 We get that value by considering that a product should appear at least in 3 transactions in a day. Our data is collected over a week. Hence, the support value should be 3*7/7500 = 0.0028</li><li>Minimum confidence, which we choose to be 0.2 (obtained over-analyzing various results)</li><li>Minimum lift, which we’ve set to 3</li><li>Minimum Length is set to 2, as we are calculating the lift values for buying an item B given another item A is bought, so we take 2 items into consideration.</li><li>Minimum Length is set to 2 using the same logic[6].</li></ol><pre>from apyori import apriori<br>rules = apriori(transactions = transactions, min_support = 0.003, min_cinfidence = 0.2, min_lift = 3, min_length = 2, max_length = 2)</pre><h4>Step-7: Print out the number of rules as list</h4><pre>results = list(rules)</pre><h4>Step-8: Have a glance at the rules</h4><pre>results</pre><figure><img alt="We print out the results as a List" src="https://cdn-images-1.medium.com/max/1024/1*9yP3WCtRQqm5omDNuqDEFg.png" /><figcaption><strong>Figure 15. We print out the results as a List</strong></figcaption></figure><figure><img alt="market basket analysis" src="https://cdn-images-1.medium.com/max/1024/1*0xyxcrJhOTncryWyJ7BJyQ.png" /><figcaption><strong>Figure 16. Market basket analysis [4]</strong></figcaption></figure><h4>Step-9: Visualizing the results</h4><p>In the <strong>LHS</strong> variable, we store the first item from all the results, from which we obtain the second item that is bought after that item is already bought, which is now stored in the <strong>RHS</strong> variable. <br> The <strong>supports, confidences and lifts</strong> store all the support, confidence and lift values from the results [6].</p><pre>def inspect(results):<br>    lhs         =[tuple(result[2][0][0])[0] for result in results]<br>    rhs         =[tuple(result[2][0][1])[0] for result in results]<br>    supports    =[result[1] for result in results]<br>    confidences =[result[2][0][2] for result in results]<br>    lifts        =[result[2][0][3] for result in results]<br>    return list (zip(lhs, rhs, supports, confidences, lifts))<br>resultsinDataFrame = pd.DataFrame(inspect(results), columns = [&quot;Left hand side&quot;, &quot;Right hand side&quot;, &quot;Support&quot;, &quot;Confidence&quot;, &quot;Lift&quot;])</pre><p>Finally, we store these variables into one dataframe, so that they are easier to visualize.</p><pre>resultsinDataFrame</pre><figure><img alt="variables into one dataframe" src="https://cdn-images-1.medium.com/max/615/1*ul8ANc0j0DRUOyY7K57R8Q.png" /><figcaption><strong>Figure 17. Variables into one dataframe</strong></figcaption></figure><p><strong>Now</strong>, we sort these final outputs in the descending order of lifts.</p><pre>resultsinDataFrame.nlargest(n = 10, columns = &quot;Lift&quot;)</pre><figure><img alt="sort these final outputs" src="https://cdn-images-1.medium.com/max/599/1*q7WRCVGcxpRTv0WFhlETMA.png" /><figcaption><strong>Figure 18. Sort these final outputs</strong></figcaption></figure><p>This is the final result of our apriori implementation in python. The SuperMarket will use this data to boost their sales and prioritize giving offers on the pair of items with greater Lift values [6].</p><h3>Why Apriori?</h3><ol><li>It is an easy-to-implement and easy-to-understand algorithm.</li><li>It can be easily implemented on large datasets.</li></ol><h3>Limitations of Apriori Algorithm</h3><p>Despite being a simple one, Apriori algorithms have some limitations including:</p><ul><li>Waste of time when it comes to handling a large number of candidates with frequent itemsets.</li><li>The efficiency of this algorithm goes down when there is a large number of transactions going on through a limited memory capacity.</li><li>Required high computation power and need to scan the entire database[4].</li></ul><h3>Summary</h3><figure><img alt="Flowchart of Apriori algorithm." src="https://cdn-images-1.medium.com/max/850/1*MIXTYPvJwc7uU01LR397jQ.png" /><figcaption><strong>Figure 19. Flowchart of Apriori algorithm[8]</strong></figcaption></figure><p>Association rule learning is a type of unsupervised learning technique that checks for the dependency of one data item on another data item and maps accordingly so that it can be more profitable. It tries to find some interesting relations or associations among the variables of the dataset. It is based on different rules to discover the interesting relations between variables in the database. The flowchart above will help summarize the entire working of the algorithm[2].</p><blockquote><a href="https://github.com/parisa-ahmadi/DataScience-MachineLearning-Projects">Github repository for whole codes</a></blockquote><h3>References:</h3><p>[1] <a href="https://www.softwaretestinghelp.com/apriori-algorithm/">https://www.softwaretestinghelp.com/apriori-algorithm/</a></p><p>[2] <a href="https://www.javatpoint.com/association-rule-learning">https://www.javatpoint.com/association-rule-learning</a></p><p>[3] <a href="https://towardsdatascience.com/underrated-machine-learning-algorithms-apriori-1b1d7a8b7bc">https://towardsdatascience.com/underrated-machine-learning- algorithms-apriori-1b1d7a8b7bc</a></p><p>[4] <a href="https://intellipaat.com/blog/data-science-apriori-algorithm/">https://intellipaat.com/blog/data-science-apriori-algorithm/</a></p><p>[5] Patterns of user involvement in experiment-driven software development, authors.(S. Yaman), (F. Fagerholm), (M. Munezero), (T.Männistö).December 2019, <a href="https://www.journals.elsevier.com/information-and-software-technology">https://www.journals.elsevier.com/information-and-software-technology</a></p><p>[6] <a href="https://djinit-ai.github.io/2020/09/22/apriori-algorithm.html#understanding-our-used-case">https://djinit-ai.github.io/2020/09/22/apriori-algorithm.html#understanding-our-used-case</a></p><p>[7] <a href="https://www.datacamp.com/tutorial/market-basket-analysis-r">https://www.datacamp.com/tutorial/market-basket-analysis-r</a></p><p>[8] <a href="https://www.researchgate.net/figure/Flowchart-of-Apriori-algorithm_fig2_351361530">https://www.researchgate.net/figure/Flowchart-of-Apriori-algorithm_fig2_351361530https://www.researchgate.net/figure/Flowchart-of-Apriori-algorithm_fig2_351361530</a></p><h3><strong>Writer: Parisan Ahmadi</strong></h3><p>Contact: <a href="https://medium.com/@parisan.ahmadi">Medium</a>, <a href="https://github.com/parisa-ahmadi">Github</a>, <a href="https://www.kaggle.com/parisanahmadi">Kaggle</a></p><ul><li><a href="https://medium.com/@parisan.ahmadi">Parisan Ahmadi - Medium</a></li><li><a href="https://github.com/parisa-ahmadi">parisa-ahmadi - Overview</a></li><li><a href="https://www.kaggle.com/parisanahmadi">Parisan Ahmadi | Expert</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9540cfc5c11a" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-solve-the-apriori-algorithm-in-a-simple-way-from-scratch-9540cfc5c11a">How to solve the Apriori algorithm in a simple way from scratch?</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>