<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by KhALeD Denfir on Medium]]></title>
        <description><![CDATA[Stories by KhALeD Denfir on Medium]]></description>
        <link>https://medium.com/@khaleddenfir?source=rss-2dcdfcbfc40c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HcI-S7h0OZq-jvatanDSAQ.jpeg</url>
            <title>Stories by KhALeD Denfir on Medium</title>
            <link>https://medium.com/@khaleddenfir?source=rss-2dcdfcbfc40c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:32:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@khaleddenfir/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Before we develop an ML system, There are two aspects we need to check:]]></title>
            <link>https://khaleddenfir.medium.com/before-we-develop-an-ml-system-there-are-two-aspects-we-need-to-check-f5ebcd03c03e?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f5ebcd03c03e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Sat, 14 Oct 2023 13:59:14 GMT</pubDate>
            <atom:updated>2023-10-14T13:59:14.648Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-sdR6otEm8qegBXC3NVDuw.jpeg" /></figure><p>Before we develop an ML system, There are two aspects we need to check:</p><p><strong>Align business and ML:</strong></p><p>Forget about data scientist’s accuracy scores, F1 scores, … business is business, if your model does not maximize the profit, then no business will want it.</p><p>If improving the accuracy from 91% to 98%, only provides 0.001% growth in the business while spending 1% of resources, then it is a loss.</p><p>We must first make sure the project is driven by business objectives, and the ML model is a proxy to reach those business goals.</p><p>Rule of thumb: if you can not frame your problem into a task that ML can solve, then you don’t need ML yet.</p><p><strong>Requirements for an ML system:</strong></p><p>Four characteristics are to be satisfies:</p><ol><li>Reliability: The system should continue to perform the correct function at the desired level of performance, even in the face of adversity (hardware or software faults). When it fails, it should not fail silently, otherwise it is not reliable.</li><li>Scalability: a system that is ran 10 times a second, does not need same server resources as when your business needs a 1000 predictions per second.</li><li>Maintainability: Make sure you have the enough resources to respond to any bugs and new features.</li><li>Adaptability: Ability to evolve when patterns change and new trends occur</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f5ebcd03c03e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Latency and throughput in machine learning]]></title>
            <link>https://khaleddenfir.medium.com/latency-and-throughput-in-machine-learning-e467faab5bf9?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e467faab5bf9</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Sun, 08 Oct 2023 09:16:27 GMT</pubDate>
            <atom:updated>2023-10-08T09:16:27.247Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*ZNKo2IKDhdUBL0AY" /></figure><p>When managing a data science project, the first thing to do is always understand the requirements from different stakeholders.</p><p>For instance we are building a hotels recommendation system, different stakeholders will have different concerns:</p><ol><li>Product team:</li><li>They would want a model that returns recommended hotels in 100 milliseconds, as 30% of increase in latency will reduce sales by 0.5% AND also they raise a concern that on average we do have 100 visitors at every given second.</li><li>Sales team: They want a model that recommends expensive hotels, as they bring more service fees.</li><li>After sales team: they want hotels that will raise less negative feedback.</li><li>Management: They want a model that maximizes the margin.</li></ol><p>Let’s see how these contradicting requirements affect our model deployment system.</p><p>To be able to build and deploy a model that satisfies these requirements, we may have to run a relatively complexe model; which consists of multiple models, each one answering to one requirement, then all these models are combined to output a shared decision about the prediction; it is called ensemble ML methods.</p><p>Now what will raise with this approach, is high latency; meaning as complexe and accurate the model can be, it will take some time to process the data then run it through different models to later on combine the results into one prediction.</p><p>If we are to expose this machine learning system as API, and it will run more than the 100 milliseconds required by our Product team, then this model can be useless in production.</p><p>In simple words, Latency is the time it takes from receiving a query to returning the result.</p><p>Another aspect to consider is Throughput, which is how many queries are processed within a specific period of time (usually 1 second)</p><p>If the systems always processes one query at a time, with a 100 milliseconds of latency for each query, then the throughput is 10 queries in 1 second, which will only cover for 10 recommendations in one second, hence if we do have 20 visitors, only 10 will get recommendation the other group will have to wait little bit longer, knowing that we have 100 visitors on average at every given second.</p><h3>Solutions:</h3><p>To optimize this combination of latency and throughput, some of useful solutions would be:</p><ul><li>Use simpler models</li><li>Quantization: involves reducing the precision of model weights and activations. This can significantly reduce the model’s size and inference time while sacrificing some accuracy.</li><li>Pruning techniques: identify and remove less important weights or neurons from your neural network, reducing model size and improving inference speed.</li><li>Utilize hardware accelerators like GPUs, TPUs, or specialized inference chips (e.g., NVIDIA TensorRT)</li><li>Batches processing rather than one sample at a time. This reduces overhead and improves throughput.</li><li>Asynchronous inference to handle multiple inference requests simultaneously. This can help reduce waiting times and improve throughput.</li></ul><h3>Conclusion</h3><p>To add to the complexity of building ML models, a data scientist / engineer will have to consider many production aspects like scalability and effectiveness, even before starting the project to make sure resources are not wasted.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e467faab5bf9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Airflow for data scientists]]></title>
            <link>https://medium.com/analytics-vidhya/airflow-for-data-scientists-1e8554d6984a?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e8554d6984a</guid>
            <category><![CDATA[etl]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[airflow]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Sat, 18 Feb 2023 16:20:43 GMT</pubDate>
            <atom:updated>2023-02-18T16:20:43.661Z</atom:updated>
            <content:encoded><![CDATA[<p>As data scientists, we collecte data, explore for patterns, build models and then test them. And in so many cases, we will need to manage the different and sometimes complex data systems pipelines workflow.</p><p>Airflow falls into this category, as per described in the Apache’s description; <strong>Airflow</strong> is a platform that lets you build and run workflows.</p><p>A workflow is represented as a DAG (a Directed Acyclic Graph), and contains individual pieces of work called Tasks, arranged with dependencies and data flows taken into account.</p><p>Definition:</p><p>Airflow is mainly constituted by DAGs and TASKs.</p><p>A DAG is a collection of tasks with schedule information. Each task can perform different work based on our requirement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*ZCTRAdrbK6PVEaUs.png" /></figure><p>In the example above a DAG is the baking pizza process, that is made of small tasks like making the crust, and the tomato sauce then baking.</p><p>These tasks may depend on each other, meaning we can only execute one task if another one has finished, example is we can only start baking the pizza if both the crust and the topping is finished and ready.</p><p>Let’s use it in an example to make things clear.</p><p>We start by declaring our DAG information first, as the DAG is the main component:</p><pre>from airflow import DAG<br>import pendulum<br><br># define the DAG<br>my_dag = DAG(<br>    dag_id=&#39;pizza_making&#39;,<br>    schedule_interval=&#39;10/ * * * *&#39;,<br>    default_args={<br>        &#39;owner&#39;: &#39;khaled&#39;,<br>        &#39;start_date&#39;: pendulum.datetime(2023, 2, 18),<br>    }<br>)</pre><p>We attribute basic parameters to the dag:</p><ul><li>dag_id: a unique alphanumeric ID for the DAG</li><li>schedule_interval: a crontab like syntax, check <a href="https://crontab.guru/">https://crontab.guru/</a></li><li>default_args</li></ul><p>We declare our Python function that will apply the first task:</p><pre>def prepare_crust():<br>  return &quot;I MADE THE BEST CHEESY CRUST EVER&quot;</pre><p>Then attach this task into the dag:</p><pre>from airflow.operators.python import PythonOperator<br><br>task1 = PythonOperator(<br>    task_id=&#39;crust_task&#39;,<br>    python_callable=prepare_crust,<br>    dag=my_dag<br>)</pre><p>Now we define our second task, and attach it to the same DAG:</p><pre>def making_sauce():<br>  reurn &quot;I MADE MY SAUCE&quot;<br><br>task2 = PythonOperator(<br>    task_id=&#39;sauce_task&#39;,<br>    python_callable=making_sauce,<br>    dag=my_dag<br>)</pre><p>And now our last task which is baking our pizza.</p><pre>def baking_pizza():<br>  reurn &quot;I BAKE MY ITALIAN PIZZA&quot;<br><br>task3 = PythonOperator(<br>    task_id=&#39;baking_task&#39;,<br>    python_callable=baking_pizza,<br>    dag=my_dag<br>)</pre><p>At this stage our tasks are ready, and the last step is defining the workflow using double &gt;&gt;</p><pre>[task1,task2] &gt;&gt; task3</pre><p>This means that task1 and task2 can be done simultaneously, while to execute task3 we need both task1 and task2 to be completed.</p><p>At the end to visualize the workflow, we will need to open the airflow interface, and explore multiple segments:</p><p><strong>Tree view of the tasks:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*K6-20hAnPqTH6IIy.png" /></figure><p><strong>Graph view of our DAG:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*af1JO1pXZRCkfrDm.png" /></figure><p>One of the aspects that is important is the ability to pass values/variables from one task to another, an example would be to pass to the baking task that cheese should only be baked for the last 30 seconds of the baking, and not the full 6 minutes.</p><p>To do that we use xcom_push and xcom_pull:</p><ul><li>xcom_push will be used to pass data to the task instance</li><li>xcom_pull will be used to pull that data and use it later</li></ul><p>Here is an example of how the functions of task1 and task3 will be:</p><pre>from airflow.models import taskinstance<br><br><br><br>def prepare_crust():<br>  cheese_baking_time_in_seconds = 30<br>  task_instance.xcom_push(<br>        key = &quot;cheese_baking_time_in_seconds&quot;, value = cheese_baking_time_in_seconds<br>    )<br>  return &quot;I MADE THE BEST CHEESY CRUST EVER&quot;<br><br><br><br>def baking_pizza():<br>  cheese_baking_time_in_seconds = task_instance.xcom_pull(<br>        key = &quot;cheese_baking_time_in_seconds&quot;, task_ids=[&quot;task1&quot;]<br>    )  <br>  # here we can use the variable in our function<br>  return &quot;I BAKE MY ITALIAN PIZZA&quot;</pre><p>The rest would still be the same.</p><p><strong>Conclusion:</strong></p><p>Airflow is vast and covers so many scenarios and has many options, here we only covered most basic but enough for a data scientist to use airflow in managing different data tasks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e8554d6984a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/airflow-for-data-scientists-1e8554d6984a">Airflow for data scientists</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A clean and tidy Python app]]></title>
            <link>https://medium.com/analytics-vidhya/a-clean-and-tidy-python-app-9463495791b1?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/9463495791b1</guid>
            <category><![CDATA[virtual-environment]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[goodproject]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Mon, 08 Aug 2022 19:43:52 GMT</pubDate>
            <atom:updated>2022-08-08T19:43:52.647Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*hy8f3sGcFwCESx42.png" /></figure><p>Here i gather some tricks and good practices when starting a new app in Python, be it the app is too simple or complex, to be scaled or not, it is good to follow widely and easy to understand guidelines for clarity and easy maintenance.</p><h3>Git is the first step</h3><p>First step is creating the repository for version control and collaboration.</p><p>Even if you work alone, at least it is good to save code.</p><p>Your first instinct, when you start to do something new, should be git init. You’re starting to write a new paper, you’re writing a bit of code to do a computer simulation, you’re mucking around with some new data … anything: think git init.</p><p>Create the repository with the name of your app an then initialize git.</p><pre>mkdir beastapp<br>git init</pre><h3>Virtual environment</h3><p>It is good for every project to have its own virtual environment, for dependencies management.</p><p>If you never use virtual environment, some projects will have different modules versions, hence you will keep uninstalling and then installing different version every time, now imagine this in a production server and the pain it is going to cause.</p><p>Above all it is super easy to create, activate and deactivate, navigate to the project folder then:</p><pre>python -m venv venv         #to create it<br>source venv/bin/activate    #to activate it<br>deactivate                  #to deactivate when finished using</pre><h3>The files skeleton</h3><p>As Python apps usually are very easy to understand compared to C or Java, the are no clear conventional way on how to organize the files/folders/modules inside the repository.</p><p>But this being said, everyone should implement good practice in his own project, hence i usually decide to go with this easy tree architecture.</p><p>When executing the tree command, we should see something like this:</p><pre>(base) khaled@khaled-T490:~/Desktop/beastapp$ tree<br>.<br>├── apidoc               #for al your api documentations<br>├── credentials          #Credentials for different remote connex<br>│   ├── es_creds.json<br>│   └── mysql_creds.json<br>├── data                 #data that won&#39;t be fetched from server<br>│   ├── data.csv<br>│   └── users.sql<br>├── doc                  #the app documentation<br>├── Readme.md            #readme including description and how to use<br>├── scripts              #for command-line interface stuff<br>├── src                  # Main app folder<br>│   └── beastapp         #another folder for the app<br>│       ├── __init__.py  # initializer app<br>│       ├── main.py     <br>│       ├── models       # Models you will be using to predict<br>│       │   ├── logisticRegression.pkl<br>│       │   ├── minmax_std.pkl<br>│       │   └── RNN_2_01.pkl<br>│       ├── mysql_con.py<br>│       └── predict.py<br>└── tests                # files/scripts used for testing the app<br>    └── functions_tester.py</pre><pre>9 directories, 13 files</pre><p>This tree is not hard coded, means feel free to adjust based on your preferences and needs, but keep it clean and tidy.</p><h3>Exclude venv from GIT and requirements</h3><p>Last thing is to exclude the venv from the repository by adding it to the .gitignore</p><p>There is no need to push the venv to git, as it may not even be usable by the cloner of the app.</p><p>But add a requirements.txt to your repository for next user to install them.</p><pre>pip freeze &gt; requirements.txt      # to generate those requirements<br>pip install -r requirements.txt    # to install them</pre><p>And that’s it to conclude, feel free to suggest anything to add/remove/edit in this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9463495791b1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/a-clean-and-tidy-python-app-9463495791b1">A clean and tidy Python app</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[First steps into contributing to open-source projects]]></title>
            <link>https://medium.com/analytics-vidhya/first-steps-into-contributing-to-open-source-projects-7f15c069af1b?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f15c069af1b</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Sun, 07 Aug 2022 11:52:02 GMT</pubDate>
            <atom:updated>2022-08-07T11:52:02.929Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*enNmLyIPiyUMmFNyZQcx2w.png" /><figcaption>OPEN SOURCE IS KEY</figcaption></figure><p>I have been project lead for quite some time, and one thing i noticed, is new folks rarely knew GIT contribution logic and commands, and that is due to non-participation into open-source projects before.</p><p>Hence i provide here quick easy way to do so in any GitHub project.</p><p>In big open-source (or even closed ones) projects you won’t get direct access to commit, push or create branches, but you will have to use eventually a pull request which has a name that explains itself.</p><p>Then a main maintainer of the project, will review the pull request and either accept, request some changes or refuse the changes.</p><p>I will use this repository as an example: <a href="https://github.com/elastic/eland">https://github.com/elastic/eland</a></p><p>Here’s a walk-through the process of submitting a pull request for an open-source project:</p><h3>Fork the repository</h3><p>Use the button in the top right of the project GitHub page, this will create/copy the project in your account.</p><p>This can also be used to copy the repository into your account and never lose it, even when project is deleted from source.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*afASIWwI8IGPpPbXLDt4Ag.png" /><figcaption>May the fork be with you</figcaption></figure><h3>Clone it</h3><p>Go to your user account repositories, there will be the new forked repos under the same name as source. Clone it into your local machine.</p><blockquote>To clone, either use HTTPS or SHH by clicking the green code button, and copy pasting the command into your shell.</blockquote><p>git clone git@github.com:swishderzy/eland.git</p><blockquote>Check this well written blog about HTTPS vs SSH cloning: <a href="https://ourtechroom.com/tech/https-vs-ssh-in-git/">https://ourtechroom.com/tech/https-vs-ssh-in-git/</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X3jBNDAlF1DqdVVnq_d22w.png" /><figcaption>Clone the repository</figcaption></figure><p>Now you will have local repository of your fork, where you can edit make changes and test.</p><h3>Synchronize it with the source/remote repository</h3><p>So that the local repository stays up-to-date with the source one, reflecting new changes. To do that:</p><blockquote>git remote add upstream <a href="https://github.com/swishderzy/eland.git">https://github.com/swishderzy/eland.git</a></blockquote><h3>Branch it up</h3><p>First we need to properly prepare for the branch.</p><p>To make sure you are on the main branch, usually the default branch is either main or master, but not always.</p><pre>git checkout master</pre><p>The fetch the latest changes using the synchronization we added before, then merge it to the default branch.</p><pre>git fetch upstream<br>git merge upstream/master<br>git push origin master</pre><p>Now we create the new branch, that usually has small representative name of your changes, it can be new feature, or bug fix.</p><pre>git checkout -b print-bug-fix</pre><p>Make changes within this branch and tested, we push the changes into your fork.</p><pre>touch bugged_file.py<br>git add bugged_file.py<br>git commit -m &quot;Fixed that annoying print bug&quot;<br>git push origin print-bug-fix</pre><h3>The pull request</h3><p>Now visit your repository on GitHub website, and click “New Pull Request” to start submitting the pull request.</p><p>Explain the changes you made, their purpose, the use case, context so that reviewers will easily understand.</p><p>Most likely reviewers will guide you later on the changes.</p><h3>Guidelines</h3><p>To improve the chances of your pull request being reviewed, you should:</p><ul><li>Reference an open issue for non-trivial changes to clarify the PR’s purpose</li><li>Ensure you have appropriate tests. These should be the first part of any PR</li><li>Keep your pull requests as simple as possible.</li><li>Ensure that CI is in a green state.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f15c069af1b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/first-steps-into-contributing-to-open-source-projects-7f15c069af1b">First steps into contributing to open-source projects</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Logistic Regression theory]]></title>
            <link>https://medium.com/analytics-vidhya/logistic-regression-theory-e62bde5d2b2c?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e62bde5d2b2c</guid>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Thu, 31 Dec 2020 12:15:23 GMT</pubDate>
            <atom:updated>2021-02-02T16:52:50.296Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://khaleddenfir.medium.com/linear-regression-theory-1736093dfd35">Logistic Regression theory recap</a></h3><p>Logistic regression is a Classification algorithm that will draw a boundary to differentiate between 2 or more classes, the boundary can be linear or non-linear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/0*F5Bta4D-HbRRt9C7.png" /><figcaption>Boundaries for classification</figcaption></figure><p>The starting logic is same as in the linear regression where we have a cost function that is the difference between the real dependent variable values and the decision boundary values.</p><p>But since this is a classification (and for more clearer explanation we consider a binary classifications), then the hypothesis function should output values between 0 and 1, and is expressed by the following Sigmoid function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/0*b51zuoqG5xWwqA8G.png" /></figure><p>The downside of this function that once it is plugged into our cost function, the cost function will not be convex (with local minima = global minima) but will have lots of local minimas, hence it is hard to catch the global minima.</p><p>To overcome this, we use the <strong>logarithm</strong> of this function in the following way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/1*bz8Jc8YgZIUg82R_g6_Qog.png" /></figure><p>Or in a more compact way, applied to all data points we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*kW9KLYx8w2VHrUNInLmaBA.png" /></figure><p>As for finding the optimal θ that will minimize this cost function, we use the <strong>gradient descent</strong> in the same way it is used for linear regression, the only difference is that the cost and hypothesis functions are different.</p><p>It is important to notice that gradient descent is far from being the only algorithm that can find optimal θ, there are many more.</p><p><strong>Multi-class classification:</strong></p><p>When our data is labeled by more than 2 classes (K classes), easiest way is to use the <strong>one vs. all</strong> method where we train K classifiers, each classifier will classify if a new data belongs to that class or not.</p><p>We will end up with K classifiers, once we have an input to classify we take the most confident classifier (the classifier that outputs the highest probability)</p><p>References:</p><ul><li>Andrew NG machine learning course on coursera</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e62bde5d2b2c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/logistic-regression-theory-e62bde5d2b2c">Logistic Regression theory</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Regression theory]]></title>
            <link>https://medium.com/analytics-vidhya/linear-regression-theory-1736093dfd35?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1736093dfd35</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[normal-equation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[linear-regression]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Wed, 30 Dec 2020 12:13:08 GMT</pubDate>
            <atom:updated>2021-01-10T14:35:51.447Z</atom:updated>
            <content:encoded><![CDATA[<h3>Linear Regression theory recap</h3><h3>Problem statement:</h3><p>To better understand the linear regression let’s see a quick example when we only have one dependent variable and one independent variable</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/0*q9C7G5zQFbiFaFlv" /><figcaption>Distances between actual Y values and linear line to minimize</figcaption></figure><p>To be able to predict Y values based on X values, we want to find the best line that will be close enough to all the points to be considered as the optimal straight line possible.</p><p>Since it is a linear relationship we are looking for, the fitting line is going to be straight , hence the line can be defined by the function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*BeKV8IFlzeXf4IKfLinfsQ.png" /></figure><p>Since in our example there is only one independent variable, then the function will be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/210/1*dIazJl07ZLnPyZIELbaPLg.png" /></figure><p>One way to express the difference between the best fitting line and the actual Y values is the<strong> least-squares cost function</strong>, defined by:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/335/1*_YGc_jfOGL_lr_ASQzCoNQ.png" /></figure><p>When using this particular cost function, then we we are solving an<strong> ordinary least squares regression model.</strong></p><p>Now our goal is to <strong>find optimal parameters (θ) that minimize this function J(θ) (the difference)</strong></p><h3><strong>Solution (Finding best fitting line):</strong></h3><ol><li><strong>Batch gradient descent:</strong></li></ol><p>One way to minimize that function and find the optimal θ is using the <strong>gradient descent</strong></p><p>The gradient descent is an iterative algorithm that will get closer to the optimal parameters with time.</p><p><strong>First: </strong>we start some random initial θ</p><p><strong>Second: </strong>we repeatedly perform this iteration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/1*EjLBpsD21alMDZZph9fagQ.png" /></figure><p><strong>Annotations:</strong></p><ul><li>a := b means assign the value of b to a</li><li>α is the learning rate (more explanation later)</li><li>∂ is the partial derivative symbol</li></ul><p><strong>Third: </strong>Let’s calculate that partial derivative for clearer iterations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/542/1*cglcgU5RSOiZrSgmJ_1HQg.png" /></figure><p>So for a single training example we get the rule:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*4LucaGuhkW3yVhfS8u134w.png" /></figure><p>This rule is called <strong>LMS (least mean squares)</strong> or <strong>Widrow-Hoff </strong>learning rule, because the magnitude of the update is proportional to the error term (y(i) − hθ (x(i) ))</p><p>Thus, for instance, if we are encountering a training example on which our prediction nearly matches the actual value of y (i) , then we find that there is little need to change the parameters; in contrast, a larger change to the parameters will be made if our prediction h θ (x(i)) has a large error (i.e., if it is very far from y(i) ).</p><p><strong>Fourth: </strong>When there are more than one single training example, the rule will be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*-CfX4H5cifQmkzhyZvCf4A.png" /></figure><p>So, this is simply gradient descent on the original cost function J.</p><p><strong>Annotation reminder:</strong></p><ul><li>The small j in the θj goes from 1 to n and denotes which parameter</li><li>The i is the data sample row</li></ul><p>This method looks at every example in the entire training set on every step, and is called batch gradient descent.</p><p>Note that, while gradient descent can be susceptible to local minima in general, the optimization problem we have posed here for linear regression has only one global, and no other local, optima; thus gradient descent always converges (assuming the learning rate α is not too large) to the global minimum. Indeed, J is a convex quadratic function.<br>Here is an example of gradient descent as it is run to minimize a quadratic<br>function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*2p0v73OCvaEOmrZa7aD2yA.png" /></figure><p>The ellipses shown above are the contours of a quadratic function. Also<br>shown is the trajectory taken by gradient descent, with was initialized at<br>(48,30). The x’s in the figure (joined by straight lines) mark the successive<br>values of θ that gradient descent went through.</p><p><strong>2. Stochastic gradient descent:</strong></p><p>Whereas batch gradient descent has to scan through the entire training set before taking a single step — a costly operation if m is large — <strong>stochastic gradient descent</strong> splits data into small chunks and on each step it trains data only on that particular chunk while looping over the steps, hence it can start making progress right away, and continues to make progress with each example it looks at.</p><p>Often, stochastic gradient descent gets θ “close” to the minimum much faster than batch gradient descent.</p><p>Note however that it may never “converge” to the minimum, and the parameters θ will keep oscillating around the minimum of J(θ); but in practice most of the values near the minimum will be reasonably good approximations to the true minimum.</p><p>For these reasons, particularly when the training set is large, stochastic gradient descent is often preferred over batch gradient descent.</p><p>The <strong>stochastic gradient descent</strong> algorithm is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*7xjLYzGk7n894KJ5xHZyBg.png" /></figure><p><strong>3. Normal equation:</strong></p><p>Another way to compute the θ that will minimize the cost function is using an Analytical method where we find the θ that will make the <strong>derivative of the cost function equal to zero.</strong></p><p>Where the θ that will minimize the cost function can be found with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/559/1*MXJvwQis1ACzWHVICKvcHg.png" /></figure><p>Where:</p><ul><li>X is a (m,n) matrix where m is the number of rows and n the columns</li><li>y is the dependent variable’s vector (the variable that we want to predict)</li></ul><p>Don&#39;t forget to add a column with [1]xm in the first columns to represent X0</p><p>The downside of this method is that it is slow when n (number of features) is large (more than 10K).</p><p>References:</p><ul><li>Andrew NG (Stanford University) ML notes.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1736093dfd35" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/linear-regression-theory-1736093dfd35">Linear Regression theory</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Intuition of word embeddings #NLP]]></title>
            <link>https://medium.com/analytics-vidhya/intuition-of-word-embeddings-nlp-8cc88314562e?source=rss-2dcdfcbfc40c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8cc88314562e</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[word-embeddings]]></category>
            <dc:creator><![CDATA[KhALeD Denfir]]></dc:creator>
            <pubDate>Mon, 22 Oct 2018 13:11:46 GMT</pubDate>
            <atom:updated>2018-10-22T13:11:46.652Z</atom:updated>
            <content:encoded><![CDATA[<h3>Intuition of word embeddings #NLP — Part 1</h3><p>Word embedding is simply assigning numerical value(s) to every word in some vocabulary. Because almost every algorithm in Machine learning (except random trees) needs numerical inputs, so puting words as inputs is not an option, some Python libraries include encodings in their packages to facilitate the process for users.</p><p>Let’s say we have the sentence: “ Who let the dogs out” (without the “hoo hoo”, haha).</p><p>to process that we have to encode every unique word with a numerical value:</p><p>Who →1</p><p>Let →2</p><p>the →3</p><p>dogs →4</p><p>out →5</p><p>So then this sentence would be expressed by a vector: [1,2,3,4,5].</p><p>Now imagine we have sentences of thousands of words, the encoding would reach enormous numbers, let’s say the first word in this long long sentence is “The” and the last word in it is “saved”, so the encoding of these twho words would be<strong> “the” →1</strong> and <strong>“saved” →1000</strong>, in this case when you feed these values into the algorithm, it will consider that “saved” is greater than “the” while we know that this is wrong, and this will compromise the whole process.</p><p>To deal with this issue, we do not assign a scalar to every word, but a vector, and better than that, we assign a <strong>one hot vector</strong> ( a vector of zeros and ones):</p><p>Then the example would be:</p><p>Who →[1,0,0,0,0]</p><p>Let →[0,1,0,0,0]</p><p>the →[0,0,1,0,0]</p><p>dogs →[0,0,0,1,0]</p><p>out →[0,0,0,0,1]</p><p>That would resolve lot of problems, but still there’s a an other issue, what if the vocabulary is huge then we would have thousands of vectors for every sentence.</p><p>A text of 100 words, and a vocabulary of 10,000 words would give us an embeding matrix of size: 100 * 10,000</p><p>Clearly We have to shrink these word vectors without losing unicity of every word representation.</p><p>Methods of <strong>dimensionality reduction</strong> would be reasonable, but there’s a much better approach, which will not only reduce the dimension of the embedding vectors, but considers meaning and semantic of words sot that words like “dog” and “cat” would have more similar vectors than “dog” and “computer”.</p><p>This approach is called <strong>Word2Vec. </strong>Where its goals are:</p><p>1- mapping words representations into a lower dimension vector (n), this involves transforming a text to an n columned matrix (count_of_words * n).</p><p>2- Maintaining word context and meaning.</p><p>It’s crucial to know that there are two process logics for Word2Vec:</p><p>1- <strong>Skip-gram</strong>: takes an input word and attempts to estimate the probability of other words appearing close to that word.</p><p>Grams: are the group of words around the concerned word, in the example above, if we take the word “the” as the concerned word then [“who”, “let”, “dogs”, “out”] are the grams around it.</p><p>Skip: is the number of times an input word is repeated in the data with different context (different grams).</p><p>2- <strong>CBOW</strong> (continuous bag of words): takes some context as input and tries to find the single word that has the highest probability of fitting that context.</p><p>To wrap out the two logics:</p><p>skip-gram takes a word as input and outputs a context (words that appear often with that input).</p><p>CBOW take as input words (contexts) and outputs the words that appear aften with these contexts.</p><p>The Word2vec approach is trained using Neural Networks, that take as input a one hot vector (like shown before) and outputs probability of every context word. After training is complete, the embedding vector of that word would be the values in the nodes of the hidden layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*b8Ij64qO64Y8yWmLmtYnOA.jpeg" /></figure><p>In this image, the hidden layer is 300 sized, so the embedding vector would be 300 sized.</p><p>Now this is the intuition for the word embeddings, i will try to do another tutorial on how to implement this in python.</p><p>Any suggestions are welcome.</p><p>Peace.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8cc88314562e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/intuition-of-word-embeddings-nlp-8cc88314562e">Intuition of word embeddings #NLP</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>