<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Daniel Hernandez on Medium]]></title>
        <description><![CDATA[Stories by Daniel Hernandez on Medium]]></description>
        <link>https://medium.com/@danhergir?source=rss-190dfb810abc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xxqpu8JMonc2zxltBw6X8Q.jpeg</url>
            <title>Stories by Daniel Hernandez on Medium</title>
            <link>https://medium.com/@danhergir?source=rss-190dfb810abc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:30:06 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@danhergir/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Following the Wires: Understanding Anthropic’s latest paper]]></title>
            <link>https://danhergir.medium.com/following-the-wires-understanding-anthropics-latest-paper-1c401861e330?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c401861e330</guid>
            <category><![CDATA[anthropic-ai]]></category>
            <category><![CDATA[ai-interpretability]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[circuit-tracing]]></category>
            <category><![CDATA[ai-research]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Wed, 30 Apr 2025 01:43:00 GMT</pubDate>
            <atom:updated>2025-04-30T01:43:00.510Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*24mUyZ_efyM2T4rI0uDsEg.png" /><figcaption>Taken from <a href="https://transformer-circuits.pub/2025/attribution-graphs/methods.html">Circuit Tracing: Revealing Computational Graphs in Language Models</a></figcaption></figure><p>LLMs (Large language Models) have become amazingly good at tasks we once thought were uniquely human. But behind their impressive performance lies a complex network of computations that can feel like a black box.</p><p>Understanding <em>why</em> a model makes a certain prediction is a crucial area of research. In Anthropic’s latest paper, a fascinating new approach called <strong>attribution graphs</strong> is offering a promising way to peek inside.</p><p>For machine learning experts, the idea of interpretability isn’t new. Various methods to understand what our models are learning have been explored. However, when it comes to the massive scale and intricate architecture of transformer models, traditional techniques often fall short. Analyzing individual neurons, for example, can be misleading as they tend to represent a jumble of different concepts.</p><p>The researchers behind <strong>attribution graphs</strong> tackle this challenge by essentially building a more transparent <strong>model</strong> for specific parts of the original <strong>transformer</strong>. Think of it like creating a simplified diagram of a small section of a complex machine to understand its immediate function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jhhZu388zPEtjtZVmfYN9g.png" /><figcaption>Transformer vs Replacement model</figcaption></figure><p><strong>Moving Beyond Neurons: Focusing on Interpretable Features</strong></p><p>A key innovation here is the shift from analyzing individual neurons to focusing on <strong>features</strong>. These <strong>features</strong> are learned by a component called a <strong>cross-layer transcoder</strong> and tend to align better with human-understandable concepts compared to single neurons that might be using multiple unrelated ideas. It’s like moving from looking at individual transistors on a chip to identifying higher-level functional units.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R1sYgn5h1hIvOAOjJ09y0Q.png" /><figcaption>Cross-layer transcoder architecture</figcaption></figure><p><strong>Visualizing the Information Flow: The Attribution Graph</strong></p><p>The core of this method lies in creating a visual representation called an <strong>attribution graph</strong> for a specific input. This graph illustrates the journey of information within the model as it processes the input to arrive at an output. Imagine a flowchart where:</p><ul><li><strong>Input Tokens:</strong> The words or parts of words you feed into the model are the starting points.</li><li><strong>Output Tokens:</strong> The model’s prediction is the end point.</li><li><strong>Intermediate Features:</strong> These are the active, interpretable <strong>features</strong> (our functional units) that the model uses during its computation.</li><li><strong>Edges (Arrows):</strong> These show the direct influence of one feature or token on another, indicating how the activation of one component contributes to the activation of the next, ultimately leading to the final prediction.</li></ul><p><strong>How Are These Graphs Constructed?</strong></p><p>Creating these graphs involves an interesting process. For a given input, a <strong>local replacement model </strong>is created that mimics the original model’s behavior for that specific input. This replacement model uses the learned features from the <strong>cross-layer transcoder</strong> instead of the original model’s internal layers. By <em>freezing</em> certain aspects of the model’s behavior (like attention patterns), they ensure that the interactions between these <strong>features</strong> become more straightforward and easier to analyze. The connections and their strengths within this <strong>replacement model</strong> then form the basis of the <strong>attribution graph</strong>.</p><p><strong>What Can We Learn?</strong></p><p>The paper provides fascinating examples of what these <strong>attribution graphs</strong> can reveal. For instance, when the model correctly completes an acronym, the graph clearly shows how <strong>features</strong> related to individual letters and the concept of an acronym activate in sequence, leading to the correct output.</p><p>In another example, the graph demonstrates how the model activates features associated with a person (like Michael Jordan) and a concept (like sports) to arrive at the answer “basketball”.</p><p>These examples highlight the potential of <strong>attribution graphs</strong> to show us:</p><ul><li>Which parts of the input are most influential in the model’s decision.</li><li>How the model processes information through different stages of abstraction.</li></ul><p><strong>How Far Are We From Full Understanding?</strong></p><p>While <strong>attribution graphs</strong> offer a significant step forward, the paper also explores some of the limitations and the distance we still need to travel to fully understand these complex models. Some challenges include:</p><ul><li><strong>Missing the Attention Story:</strong> Currently, <strong>attribution graphs</strong> don’t fully capture the crucial role of attention mechanisms, which allow the model to focus on relevant parts of the input. This means we might be missing a key part of the puzzle in understanding how the model decides what to pay attention to.</li><li><strong>Unexplained Components:</strong> The replacement model isn’t a perfect replica, and some aspects of the original model’s behavior remain unexplained, represented as <strong>error nodes</strong> in the graphs. This dark matter in the model’s computations still needs to be understood.</li></ul><p><strong>Conclusion: A Step Towards Transparency</strong></p><p><strong>Attribution graphs</strong> represent an exciting advancement in our quest to understand the inner workings of large language models. By providing a more interpretable view of the computational steps involved in generating an output, they offer valuable insights into the model’s reasoning process. While significant challenges remain in achieving a complete understanding, this method provides a powerful new lens through which we can begin to unravel the mysteries within the black box of neural networks. As research in this area continues to evolve, we can expect even more sophisticated tools to emerge, bringing us closer to truly understanding the intelligence we are building.</p><p><strong>Sources and related content:</strong></p><p><strong>Anthropic’s Circuit Tracing Paper: </strong><a href="https://transformer-circuits.pub/2025/attribution-graphs/methods.html">https://transformer-circuits.pub/2025/attribution-graphs/methods.html</a></p><p><strong>Anthropic: Circuit Tracing + On the Biology of a Large Language Model: </strong><a href="https://youtu.be/ig5RNJJaFJE?si=3-0fCcapTIggXdeh">https://youtu.be/ig5RNJJaFJE?si=3-0fCcapTIggXdeh</a></p><p><em>Personally, this last video helped me a lot to comprehend many of the abstract concepts, challenges, and paradoxes behind AI interpretability.</em></p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="https://danhergir.github.io/"><strong><em>https://danhergir.github.io/</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on X @ </em><a href="http://twitter.com/danhergir"><strong><em>https://x.com/danhergir</em></strong></a><strong><em>.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c401861e330" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Rating Illusion: Why Your Stars Might Be Lying to You]]></title>
            <link>https://medium.com/dataplicada/the-rating-illusion-why-your-stars-might-be-lying-to-you-5bc4c7848739?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/5bc4c7848739</guid>
            <category><![CDATA[feedback]]></category>
            <category><![CDATA[confirmation-bias]]></category>
            <category><![CDATA[reviews-tool]]></category>
            <category><![CDATA[saas]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Sat, 12 Apr 2025 20:49:59 GMT</pubDate>
            <atom:updated>2025-04-12T22:36:28.464Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6VSANHtCY9OQq1T6YlzhyA.png" /></figure><p>If you’ve scrolled through Google Maps lately, you’ve probably noticed something odd: nearly every restaurant, hotel, or cafe has a glowing rating somewhere between 4.5 and 4.9 stars.</p><p>Sounds great, right?</p><p>Not exactly.</p><p>The reason? Review deletions, cherry-picked feedback, pay-to-play systems, and worst of all — <strong>confirmation bias</strong>.</p><h3>The Problem With the Stars ✨</h3><p>Let’s be real: a 4.7-star rating doesn’t mean excellence anymore — it means “average in a skewed world.” Platforms like Google Maps now allow businesses to report and remove negative reviews. That one customer who had a terrible experience? Gone. That recurring complaint about cleanliness? Buried. That 2-star rating? Magically disappears.</p><p>Suddenly, a business with a history of inconsistency looks like a Michelin-starred wonderland. And customers? They walk in expecting gold, and walk out disappointed.</p><p>The result? You don’t just lose trust. You lose <strong>repeat business</strong>.</p><h3>The Confirmation Bias Trap</h3><p>Traditional review systems are deeply flawed because they feed into a powerful psychological trap: <strong>confirmation bias</strong>. We all want to believe we’re doing great, so we selectively listen to the good and ignore the bad. It’s human nature. But it’s terrible for business.</p><p>If you only hear praise, you’ll never know what’s <em>really</em> going wrong. And by the time the truth reaches you — through churn, silent exits, or public backlash — it’s already too late.</p><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//x.com/levelsio/status/1911090771228344467&amp;image=" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bb7f3d6a3d9c92496f50165f9af2c33/href">https://medium.com/media/0bb7f3d6a3d9c92496f50165f9af2c33/href</a></iframe><h3>Feedbacker.cx: Your Unfiltered Reality Check</h3><p>At <strong>Feedbacker.cx</strong>, we believe in giving businesses the full picture — not just the flattering parts.</p><p>We collect and analyze customer feedback without bias, manipulation, or moderation filters that warp the truth. Using advanced AI and text analytics, we map the <strong>emotional tone, themes, and sentiment</strong> of your reviews to give you a dashboard that doesn’t sugarcoat your weaknesses — it spotlights them.</p><p><strong>Why?</strong></p><p>Because you can’t fix what you don’t see. Because a 4.7-star illusion won’t save you from a 1-star reality.</p><h3>Beyond Stars: Rebuilding Trust in Feedback</h3><p>Inspired by projects like <em>Hotel List</em> that are tackling rating inflation, Feedbacker.cx goes even deeper — not just normalizing ratings, but making them meaningful. We:</p><p>✅ Remove confirmation bias<br>✅ Surface underrepresented negative experiences<br>✅ Help you understand <em>why</em> customers are unhappy<br>✅ Offer data-driven insights to act before it’s too late</p><p>Let’s build businesses people actually love — not just ones that <em>look</em> like they’re loved.</p><p>🔍 <a href="http://feedbacker.cx/?utm_campaign=feedbacker_reviews&amp;utm_medium=ratingillusion&amp;utm_source=medium">Discover Feedbacker.cx</a> — Know What Your Customers <em>Really</em> Think.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5bc4c7848739" width="1" height="1" alt=""><hr><p><a href="https://medium.com/dataplicada/the-rating-illusion-why-your-stars-might-be-lying-to-you-5bc4c7848739">The Rating Illusion: Why Your Stars Might Be Lying to You</a> was originally published in <a href="https://medium.com/dataplicada">Dataplicada Insights</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Effortless Ubuntu Server Deployments with GitHub Actions ⚙️]]></title>
            <link>https://blog.devops.dev/effortless-ubuntu-server-deployments-with-github-actions-%EF%B8%8F-3ded9ea699e8?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ded9ea699e8</guid>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Wed, 25 Sep 2024 05:15:27 GMT</pubDate>
            <atom:updated>2024-09-26T17:21:34.026Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IZp0dA0sMSOqRRIk" /></figure><p>In this article, I’m not aiming to explain anything groundbreaking to experts. My goal is to help beginners automate simple tasks, like pushing or pulling code to GitHub — something I found myself doing regularly. For instance, I would frequently push to a development branch, then merge it into my main branch, log into my server, pull the latest changes, and run the necessary commands to update my project with the new changes.</p><p>This whole process might only take 5 minutes, but doing it day in and day out made me realize it was something I could automate. It would not only save time but also set the foundation for scaling up with features like unit testing, which would enhance my continuous integration and deployment (CI/CD) workflow.</p><p>For this, I decided to use GitHub Actions, which is a CI/CD tool provided by GitHub. It allows you to automate tasks like building, testing, and deploying code directly from your GitHub repository. By defining workflows in a YAML file, you can automate tasks triggered by specific events, like pushing code to a branch or opening a pull request. This saves time and helps ensure consistency across your development process.</p><p>Okay, so what if we start to automate things now? In this article, I am hoping to show you how to run certain actions after having done a git push to your main branch in GitHub.</p><p>We’ll do it step-by-step:</p><h4>Let’s create our Actions Secrets:</h4><p>Before starting to create our YML file where we are going to add the instructions and commands we want to send, we’ll have to create some secrets with the information of our server.</p><ol><li>Open your repository</li><li>Go to settings -&gt; Secrets and variables -&gt; Actions</li></ol><p>Here to create the Actions Secrets, we’ll have to add:</p><ul><li><strong>SERVER_USER</strong>: which tends to be root as the default user.</li><li><strong>SERVER_IP</strong>: Your Ubuntu server IP</li><li><strong>APP_DIR</strong>: Directory path where you app is located, normally it is /var/www/website</li><li><strong>SSH_PRIVATE_KEY</strong>: Private SSH keys generated by your server.</li></ul><h4>Generating SSH Keys:</h4><p>I’d like to explain the SSH keys generation process very well here because this could give you a hard time. Now, go to your server and run. Leave this directory</p><pre>ssh-keygen -t rsa -b 4096 -C &#39;youremail@mail.com&#39;</pre><p>Press <strong>Enter</strong> to accept the default file location ~/.ssh/id_rsa.</p><p>When prompted, you can add a passphrase (optional, but recommended for extra security.</p><p>Then, run:</p><pre>cat ~/.ssh/id_rsa.pub</pre><p>This will display your <strong>public key</strong>, which you’ll copy for the next step.</p><ol><li>Go to your GitHub account and navigate to <strong>Settings</strong>.</li><li>Under <strong>Access</strong> (on the left sidebar), click <strong>SSH and GPG keys</strong>.</li><li>Click <strong>New SSH Key</strong>.</li><li>Give the key a descriptive title (e.g., “Ubuntu Server”).</li><li>Paste the public key that you copied from the droplet ~/.ssh/id_rsa.pub.</li><li>Click <strong>Add SSH Key</strong>.</li></ol><p>Now, for the <strong>private key</strong>, run:</p><p>Notice the public key has a <strong>.pub </strong>extension, the private does not have one.</p><pre>cat ~/.ssh/id_rsa</pre><p>This will display your <strong>private key</strong>, which you’ll copy for the next steps.</p><ol><li>Go to your GitHub repository and navigate to <strong>Settings</strong>.</li><li>Under <strong>Secrets and variables</strong> click on <strong>Actions</strong>.</li><li>Click <strong>New repository secret</strong>.</li><li>Name it <strong>SSH_PRIVATE_KEY</strong>.</li><li>Paste the public key that you copied from the droplet ~/.ssh/id_rsa.</li><li>Make sure that your SSH private key is in the correct format. It should look something like this. Copy the whole code, including these lines.</li></ol><pre>-----BEGIN OPENSSH PRIVATE KEY-----<br>...<br>-----END OPENSSH PRIVATE KEY-----</pre><p>This is how your secrets should look like:</p><figure><img alt="Action Secrets" src="https://cdn-images-1.medium.com/max/1024/1*9NL1XHbxK_9SJfQF94Uarg.png" /><figcaption>Action Secrets</figcaption></figure><h4>Adding our YML file</h4><p>Now, this is where it all makes sense.</p><p>We are going to add the YAML file below in the directory at the root of our project .github/workflows/deploy.yml .</p><pre>name: Deploy Ubuntu Server Project<br><br>on:<br>  push:<br>    branches:<br>      - main  # Change to your main branch<br><br>jobs:<br>  deploy:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v2<br>        <br>      - name: Add SSH private key to agent<br>        uses: webfactory/ssh-agent@v0.5.3<br>        with:<br>          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}  # Ensure this secret contains the new private key<br><br>      - name: Deploy to Server<br>        env:<br>          SERVER_USER: ${{ secrets.SERVER_USER }}<br>          SERVER_IP: ${{ secrets.SERVER_IP }}<br>          APP_DIR: ${{ secrets.APP_DIR }}<br>        run: |<br>          ssh -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa $SERVER_USER@$SERVER_IP &quot;<br>          cd $APP_DIR &amp;&amp;<br>          git remote set-url origin git@github.com:yourgithubuser/yourgithubrepo.git &amp;&amp;  # Dont forget to change this!!!<br>          git pull origin main &amp;&amp;<br>          npm install &amp;&amp;<br>          npm run build<br>          &quot;</pre><p>This YAML file defines a <strong>GitHub Actions workflow</strong> for deploying a project to a remote server. Here’s what each part does:</p><p><strong>1. Trigger</strong>:<br> — The workflow runs whenever you push to the `main` branch.</p><p><strong>2. Job Setup</strong>:<br> — It uses `ubuntu-latest` as the runner environment.</p><p><strong>3. Steps</strong>:<br> — <strong>Checkout code</strong>: Fetches the latest code from the repository.<br> — <strong>Add SSH key</strong>: Loads the SSH private key (stored as a GitHub secret) into the SSH agent for secure access to your server.<br> — <strong>Deploy to Server</strong>: <br> — Connects to the server using SSH.<br> — Navigates to the app directory.<br> — Pulls the latest changes from the repository.<br> — Installs dependencies with `composer` and `npm`.<br> — Builds the assets using `npm run build`.</p><p>This automates the deploy of the app whenever you push to main</p><p>From now, all you have to do is to push a new commit. To check how this works, just click on the tab <strong>Actions</strong> of your repository at the top.</p><p>Hopefully, you will be able to visualize something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L4wwWY5L7uyDXZ_OA-xCSA.png" /><figcaption>Deployment completed using GitHub Actions</figcaption></figure><p>I hope this short tutorial works for you and can help you to save some time deploying those awesome apps you are building. I’d love to hear more about the projects all of you’re building and contribute if you need a hand.</p><blockquote>To learn more about me and discover additional insights <strong>visit </strong><a href="http://danhergir.com/"><strong>https://danhergir.com</strong></a>. You can also explore my Medium articles by <strong>visiting </strong><a href="https://danhergir.medium.com/"><strong>https://danhergir.medium.com</strong></a><strong> </strong>for more in-depth content or connect with me on X @ <a href="http://twitter.com/danhergir"><strong>https://x.com/danhergir</strong></a><strong>.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ded9ea699e8" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/effortless-ubuntu-server-deployments-with-github-actions-%EF%B8%8F-3ded9ea699e8">Effortless Ubuntu Server Deployments with GitHub Actions ⚙️</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building an AI Agent Fed with DB Data using LangChain + GPT-4 ]]></title>
            <link>https://python.plainenglish.io/building-an-ai-agent-fed-with-db-data-using-langchain-gpt-4-ef2d42af820a?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef2d42af820a</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[sqlalchemy]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[gpt-4]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Sat, 22 Jun 2024 03:34:49 GMT</pubDate>
            <atom:updated>2024-06-24T04:29:07.032Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-FGOq2Hj9EwogDx1" /><figcaption>Hi there!</figcaption></figure><p>I know there are countless tutorials on different social media platforms for developing your own AI Agent, but my personal goal is to make this a definitive guide that enables you to initiate a conversation with your data.</p><p>One of the biggest problems I found in various tutorials when building an AI Agent was that if you fed your chatbot with your own data, you had to use frameworks that would limit how the agent behaved much further down the line.</p><p>In this case, we will be using LangChain, which allows us to connect and communicate with various LLMs, including the GPTs that we will be using here.</p><p>LangChain has several frameworks that allow us to tell our AI Agent how we want it to receive the data, whether it’s from an SQL query or by passing it a Pandas DataFrame.</p><p><strong>The Struggle.</strong></p><p>As I mentioned before, the biggest challenge I had was to retrieve this data and not making the Agent to be limited to how the data was obtained.</p><p>For example, I tried to use this <a href="https://api.python.langchain.com/en/latest/agents/langchain_experimental.agents.agent_toolkits.pandas.base.create_pandas_dataframe_agent.html">LangChain experimental Agent</a> which would allow you to work with a Pandas DataFrame, and it works very well reading and understanding the data, the problem comes when you want it answer naturally, but it doesn’t really matter how hard you try or how good are your instructions, it will use Pandas keywords to express opinions, and it won’t be very useful if your purpose is to make this agent to talk to a person with no expertise in Data Science.</p><p><strong>The Solution.</strong></p><p>Use a better template. That’s it, and try to avoid experimental agents, it just won’t work.</p><p><strong>Some context.</strong></p><p>Okay, so here’s what I tried to achieve. I am a Data Science Specialist and CTO at <a href="https://dataplicada.com/">Dataplicada</a>, a company dedicated to providing Data Science solutions. We are currently working on creating a service that allows small and medium-sized companies to easily obtain feedback from their customers. Our service, <a href="https://feedbacker.cx/">feedbacker.cx</a>, aims to empower our customers with a virtual data analyst available 24 hours a day, 365 days a year, providing insights and ideas about customer opinions and addressing requests, petitions, and complaints.</p><p><strong>Here’s what I tried.</strong></p><p>First, I started by obtaining the data:</p><pre>import sqlalchemy<br>import pandas as pd<br><br>database_username = &quot;username&quot;<br>database_password = &quot;password&quot;<br>database_ip = &quot;host&quot;<br>database_name = &quot;db_name&quot;<br>database_port = &quot;db_port&quot;<br><br>database_connection = sqlalchemy.create_engine(<br>        &quot;mysql+mysqlconnector://{0}:{1}@{2}:{3}/{4}&quot;.format(<br>        database_username, database_password, database_ip, database_port, database_name))<br><br>def get_comments():<br>    comments = pd.read_sql_query(<br>        &quot;&quot;&quot;SELECT * FROM all_star_comment&quot;&quot;&quot;, con=database_connection<br>    )<br><br>    comments = comments[[&#39;date&#39;, &#39;stars&#39;, &#39;comment&#39;]]<br>    return comments</pre><p>You can see, I am retrieving all the comments from my database, which is customers feedback. You can try to retrieve any kind of text data you have. It doesn’t necessarily has to be DB data, it can be a CSV, after all, I am converting my data into a Pandas DataFrame.</p><p>The next step is to build the structure of our Agent:</p><pre>from langchain_core.messages import SystemMessage<br>from langchain.chains import LLMChain<br>from langchain_core.prompts import (<br>    ChatPromptTemplate,<br>    HumanMessagePromptTemplate,<br>    MessagesPlaceholder,<br>)<br>from langchain_openai import ChatOpenAI<br><br>OPENAI_API_KEY = &#39;YOUR-OPEN-AI-API-KEY&#39;<br><br>    # Obtaining the data this agent is going to work with<br>    comments = get_comments()<br>       <br>    data_description = comments.to_string(index=False)<br><br>    prompt = ChatPromptTemplate.from_messages(<br>        [<br>            SystemMessage(<br>            content=f&quot;&quot;&quot;Answer using this data {data_description}.&quot;&quot;&quot;<br>            ),  # The persistent system prompt<br>            HumanMessagePromptTemplate.from_template(<br>                &quot;{human_input}&quot;<br>            ),  # Where the human input will injected<br>        ]<br>    )<br>    <br>    llm = ChatOpenAI(model=&#39;gpt-4-turbo-preview&#39;, temperature=0)<br><br>    chat_llm_chain = LLMChain(<br>        llm=llm,<br>        prompt=prompt,<br>        verbose=True,<br>    )<br><br>    output = chat_llm_chain.predict(human_input=&#39;How can I improve my business?&#39;)<br><br>    return output</pre><p>You can see how in this code, I am converting all the comments and their columns into a single-line string. This will provide enough context for our AI Agent.</p><p>Here’s how it looks like from my FastAPI app. And yes, all of my data is related to comments about a hotel. The most interesting thing is that all of the comments are in Spanish 🇪🇸, and it works very well after making questions in a different language.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*os-iAAQmz0VnK3NHMPzMvw.png" /><figcaption>FastAPI app</figcaption></figure><p><strong>In conclusion,</strong> don’t limit yourself to using data solely from a database. As I mentioned earlier, you can use any kind of structure that you can later transform into a Pandas DataFrame.</p><p>I hope this short tutorial works for you and can help you to deploy that awesome AI Agent you’re building. I’d love to hear about the projects you’re building and contribute if you need a hand.</p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a><strong><em>.</em></strong></blockquote><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef2d42af820a" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/building-an-ai-agent-fed-with-db-data-using-langchain-gpt-4-ef2d42af820a">Building an AI Agent Fed with DB Data using LangChain + GPT-4 🦜</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting Up FastAPI API on a Subdomain with Apache + Uvicorn]]></title>
            <link>https://python.plainenglish.io/setting-up-fastapi-api-on-a-subdomain-with-apache-uvicorn-f4acf0a85f25?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4acf0a85f25</guid>
            <category><![CDATA[apache]]></category>
            <category><![CDATA[fastapi]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[uvicorn]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Tue, 16 Apr 2024 03:23:16 GMT</pubDate>
            <atom:updated>2024-04-16T06:05:44.962Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w9QLGNYixt7u5s87" /></figure><p>The past week, I was trying to deploy an API on a subdomain of a DigitalOcean Droplet that I currently have in use, where I am running the frontend and backend of a web app.</p><p>When attempting to deploy this API, I found a significant lack of documentation related to deploying on subdomains using Apache, so I had to do some experimentation and browse different forums.</p><p>To start, my API was running using a web server called <strong>Uvicorn</strong> that allowed me to run it on my localhost, so basically what I did was point this process created by <strong>Uvicorn</strong> to a port within my server on DigitalOcean, and then using <strong>PM2</strong>, a daemon process manager, I was able to keep my API alive and then create a subdomain within my droplet on DigitalOcean for that specific port.</p><h4>Let’s do it step-by-step 🚀</h4><p><strong>1. Creating a Subdomain Record</strong></p><p>First things first, create a new <strong>A</strong> record on your domain with a hostname (e.g api.domain.com) which directs to the droplet IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uHaZ6osjvp-yjcXCSP3xhA.png" /></figure><blockquote>For the sake of this example, I am going to assume that you already have your FastAPI app in your server and Apache2 installed. In my case, my API is located in the directory /var/www/sampleapi</blockquote><p><strong>2. Updating Apache2 Config</strong></p><p>Connect to your server via SSH.</p><p>Run the following command:</p><p>nano /etc/apache2/sites-available/api.domain.com.conf</p><p>Past the code below and make sure to replace it with you own data.</p><pre>&lt;VirtualHost *:80&gt;<br>    ServerName api.domain.com<br>    ServerAdmin webmaster@localhost<br><br>    ProxyPreserveHost On<br>    ProxyPass / http://localhost:5000/<br>    ProxyPassReverse / http://localhost:5000/<br><br>    ErrorLog ${APACHE_LOG_DIR}/error.log<br>    CustomLog ${APACHE_LOG_DIR}/access.log combined<br>RewriteEngine on<br>RewriteCond %{SERVER_NAME} =api.domain.com<br>RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]<br>&lt;/VirtualHost&gt;</pre><p>Basically whatever is running in the port 5000is going to be captured by our <strong>Apache2</strong> configuration and be served by our subdomain.</p><p>Finally, restart Apache sudo systemctl restart apache2.service .</p><p><strong>3. Setting up PM2 Daemon</strong></p><p>To install <strong>PM2 </strong>in your server is required to have Node.js.</p><p>To install Node.js sudo apt install nodejs</p><p>Then install PM2 running sudo npm install pm2@latest -g</p><p>Assuming your project has all its dependencies installed including <strong>uvicorn</strong>, all we need to do now is running our <strong>PM2 Daemon.</strong></p><p>First, let’s move to our API directory cd /var/www/sampleapi</p><p>Once we’re there, let’s run the following command.</p><pre>pm2 start &quot;uvicorn main:app --reload --port 5000 --host 0.0.0.0&quot; --name sampleapi</pre><p>To check the status of our server, we can run the following command:</p><pre>pm2 l</pre><p>Hopefully, you will something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wk11Tj_uR1vOocn0cK25uw.png" /></figure><p><strong>4. Troubleshooting (Just in case)</strong></p><p>Here are some commands you can use to check the status of your server and see what’s going on:</p><ol><li>sudo systemctl status apache2: Checks the status of the Apache2 service using systemd.</li><li>sudo apachectl configtest: Verifies the syntax of Apache configuration files for errors.</li><li>pm2 log: Displays the logs of processes managed by PM2 in real-time.</li></ol><h4>Conclusion</h4><p>I hope this short tutorial works for you and can help you to deploy that awesome API you’re building.</p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a></blockquote><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4acf0a85f25" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/setting-up-fastapi-api-on-a-subdomain-with-apache-uvicorn-f4acf0a85f25">Setting Up FastAPI API on a Subdomain with Apache + Uvicorn👷🏽🦄</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Binary Classification with Neural Networks using Tensorflow & Keras ⭐️ Pt.2]]></title>
            <link>https://python.plainenglish.io/binary-classification-with-neural-networks-using-tensorflow-keras-%EF%B8%8F-pt-2-6831978765cb?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/6831978765cb</guid>
            <category><![CDATA[regularization]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[dropout-regularization]]></category>
            <category><![CDATA[binary-classification]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Wed, 20 Mar 2024 07:16:17 GMT</pubDate>
            <atom:updated>2024-03-21T00:44:06.390Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*blPAglMI917gaQc1" /></figure><p>This reading is a continuation of <a href="https://medium.com/python-in-plain-english/binary-classification-with-neural-networks-using-tensorflow-keras-412a32e75075"><strong>Binary Classification with Neural Networks using TensorFlow &amp; Keras</strong></a><strong>. </strong>I invite you to read it first to obtain a better understanding of the model we’re trying to build and how we can improve it in this new opportunity.</p><h4><strong>Going back…</strong></h4><p>The last time, we trained our model, we ended up with the following results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/0*Of-wcy0Imdg2Q9-8.png" /><figcaption>Training vs Validation</figcaption></figure><p>Here we appreciate how our model gets a better loss function goes down after every epoch, but still there’s a great distance from our validation. This tells us immediately that our model is falling in a overfitting problem.</p><p>We’re going to try with two special techniques to avoid this issue and improve the performance of this model, these are <strong>Regularizers </strong>and <strong>Dropout</strong>.</p><blockquote><em>I would like to emphasize that we will be using the same code from our past reading. Here we’ll be just creating new model architectures. Training and compiling arguments are explained in the past reading as well 👀.</em></blockquote><p>Our original model looked like this, an input layer, a hidden layer, and an output layer which obtained these results:</p><p><em>loss: 0.6252 — accuracy: 0.8614</em></p><pre>model = models.Sequential()<br># Input Layer<br>model.add(layers.Dense(16, activation=&#39;relu&#39;, input_shape=(10000,)))<br># Hidden Layer<br>model.add(layers.Dense(16, activation=&#39;relu&#39;))<br># Output Layer<br>model.add(layers.Dense(1, activation=&#39;sigmoid&#39;))</pre><h4><strong>Training with Regularizers 🔩</strong></h4><p>A regularizer is a technique used in machine learning to prevent overfitting by adding a penalty term to the objective function. This penalty encourages simpler models or smoother parameter values, ultimately improving generalization to unseen data.</p><p>Here we use the same layers, but in our hidden layer, an L2 Regularization is added with a strenght of 0.001. This means that during training, the model will not only aim to minimize the loss on the training data but also ensure that the weights of this layer remain small, effectively preventing overfitting.</p><p>What you have to understand is that this penalty encourages smaller weight values by penalizing large ones.</p><pre>from keras import regularizers<br><br>model_reg = models.Sequential()<br><br># Input Layer<br>model_reg.add(layers.Dense(16, activation=&#39;relu&#39;, input_shape=(10000,),kernel_regularizer=regularizers.l2(0.001)))<br><br># Hidden Layer<br>model_reg.add(layers.Dense(16, activation=&#39;relu&#39;, kernel_regularizer=regularizers.l2(0.001)))<br><br># Output Layer<br>model_reg.add(layers.Dense(1, activation=&#39;sigmoid&#39;))<br><br><br># Compile our model<br>model_reg.compile(optimizer=&#39;rmsprop&#39;, loss=&#39;binary_crossentropy&#39;, metrics=[&#39;acc&#39;])<br><br># Train our model<br>history_reg = model_reg.fit(partial_x_train,<br>                   partial_y_train,<br>                   epochs=20,<br>                   batch_size=512,<br>                   validation_data=(x_val,y_val))</pre><p>Now, if we want to evaluate these results, we can compare it to the loss values of our original model.</p><pre># Original loss values<br>history_dict = history.history<br>loss_values = history_dict[&#39;loss&#39;]<br>val_loss_values = history_dict[&#39;val_loss&#39;]<br><br># Dropout loss values<br>val_loss_values_reg = history_reg.history[&#39;val_loss&#39;]<br><br># Data Viz<br>fig = plt.figure(figsize=(10,10))<br>epoch = range(1,len(loss_values)+1)<br>plt.plot(epoch,val_loss_values_reg, &#39;o&#39;,label=&#39;dropout&#39;)<br>plt.plot(epoch,val_loss_values, &#39;--&#39;,label=&#39;original&#39;)<br>plt.legend()<br>plt.show()</pre><p>We obtain a model with a higher accuracy, but still not as good loss values🥴.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*QJcEQU6Pq8k7UxNWNQyAWw.png" /><figcaption>Original vs Regularization. Seems like our loss values are not good enough</figcaption></figure><p>Check out the model evaluation running this code:</p><pre>model_reg.evaluate(x_test, y_test)</pre><p>Seems like we will have to keep trying…</p><h4><strong>Training with Dropout 🔨</strong></h4><p>This technique randomly disconnects some neurons (units) in a neural network during training. It helps prevent overfitting by forcing the network to learn more robust features, as it can’t rely on any one neuron.</p><p>We’re going to transform our original model. Here we add a Dropout layer that randomly ignores 50% of the connections between the neurons in the two hidden layers and the output layer in each backpropagation pass.</p><pre># Model architecture<br>model_dropout = models.Sequential()<br><br>model_dropout.add(layers.Dense(16, activation=&#39;relu&#39;, input_shape=(10000,)))<br>model_dropout.add(layers.Dropout(0.5))<br><br>model_dropout.add(layers.Dense(16, activation=&#39;relu&#39;))<br>model_dropout.add(layers.Dropout(0.5))<br><br>model_dropout.add(layers.Dense(1, activation=&#39;sigmoid&#39;))<br><br><br># Compile our model<br>model_dropout.compile(optimizer=&#39;rmsprop&#39;, loss=&#39;binary_crossentropy&#39;, metrics=[&#39;acc&#39;])<br><br><br># Train our model<br>history_dropout = model_dropout.fit(partial_x_train,<br>                   partial_y_train,<br>                   epochs=20,<br>                   batch_size=512,<br>                   validation_data=(x_val,y_val))</pre><p>Check out the model evaluation running this code:</p><pre># Loss values of Dropout Model<br>val_loss_values = history_dict[&#39;val_loss&#39;]<br>val_loss_values_dropout = history_dropout.history[&#39;val_loss&#39;]<br><br>fig = plt.figure(figsize=(10,10))<br>epoch = range(1,len(loss_values)+1)<br>plt.plot(epoch,val_loss_values_dropout, &#39;--&#39;,label=&#39;dropout&#39;)<br>plt.plot(epoch,val_loss_values, &#39;--&#39;,label=&#39;original&#39;)<br>plt.legend()<br>plt.show()<br><br>model_dropout.evaluate(x_test, y_test)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*UDsnsWQ2bu5vG47IAOHfug.png" /><figcaption>Original vs Dropout</figcaption></figure><p>Now we can definitely notice that there was an improvement in our loss function values 🥳.</p><h4>Conclusion</h4><p>These were my final results:</p><p><strong>Regularization model:</strong> <em>loss: 0.4676 — accuracy: 0.8624</em></p><p><strong>Dropout model:</strong> <em>loss: 0.4881 — accuracy: 0.8772</em></p><p>We could obtain a model with a higher accuracy and no overfitting as our original model. In addition to this you learned how to use two different techniques to obtain a better performance with your neural networks.</p><p>I would like to invite you to explore more about <strong>Regularization</strong> and <strong>Dropout </strong>techniques, try different values for the Dropout layers and see what values work better. You can make some research as well about other kernel_regularizers such as L1 or Elastic Net.</p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a><strong><em>.</em></strong></blockquote><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a><strong> | </strong><a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://stackademic.com/"><strong>Stackademic</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6831978765cb" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/binary-classification-with-neural-networks-using-tensorflow-keras-%EF%B8%8F-pt-2-6831978765cb">Binary Classification with Neural Networks using Tensorflow &amp; Keras ⭐️ Pt.2</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Binary Classification with Neural Networks using Tensorflow & Keras ]]></title>
            <link>https://python.plainenglish.io/binary-classification-with-neural-networks-using-tensorflow-keras-412a32e75075?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/412a32e75075</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Fri, 08 Mar 2024 08:25:23 GMT</pubDate>
            <atom:updated>2024-03-11T04:27:52.316Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dUdCc9NmvCqzBFsi" /></figure><p>In this opportunity, you are going to build a neural network to make a binary classification for reviews from IMDB movies. Basically, it’s going to tell us if a review is positive or negative.</p><p>Before starting to talk more about our model, I would like to introduce some of the concepts that we’re going to be using in this article.</p><p><strong>Context ✨</strong></p><p><strong>Neural networks</strong> are computational models inspired by the structure and function of the human brain 🧠. They consist of interconnected nodes, called neurons, organized into layers. Each neuron receives inputs, performs a computation, and passes its output to the neurons in the next layer.</p><p>Our <strong>network architecture</strong> consists of:</p><ul><li>Input Layer: Receives the input data.</li><li>Hidden Layers: Intermediate layers between the input and output layers. Each hidden layer performs a set of transformations on the input data.</li><li>Output Layer: Produces the final output of the network.</li></ul><p><strong>Activation functions</strong> introduce non-linearity into the output of each neuron in a neural network. They allow neural networks to learn complex patterns and relationships in the data. When we talk about non-linearity we’re talking about avoiding to find simple correlations, but complex ones.</p><p>For our network, we will be using a ReLU and Sigmoid functions.</p><p>Why these functions? Well, ReLU works speeds up our process due to its simplicity, and Sigmoid always returns 0 or 1, and that’s perfect for our case because since it’s a binary classification, we want to know if something is either true or false.</p><p>Alright, I guess this is good enough in order for you to start to code.</p><p>Guess what? It’s <strong>Coding time ⏰</strong></p><p>Let’s import some dependencies:</p><p>In the code below <a href="https://keras.io/getting_started/">Keras</a> is being imported, Keras is an API that will help us to obtain the data we’re going to use for this project and all the tools we’ll need to build our model.</p><p>Basically, we’ll be using all the <strong>Tensorflow</strong> capabilities to build our model. Now, Tensorflow is a framework created by Google to build deep learning models which is exactly what we’re about to do.</p><pre>from keras.datasets import imdb<br>from keras import models, layers, optimizers<br>import numpy as np</pre><p>Let’s download the data:</p><pre>(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)</pre><p>Now, let’s switch our keys and values. This will make it comfortable to work with the data in the next steps:</p><pre>word_index = imdb.get_word_index()<br>word_index = dict([(value, key )for (key, value) in word_index.items()])</pre><p>Printing some words. We use the fourth index since the first three are reserved words.</p><pre>for _ in train_data[0]:<br>    # The first three indexes are reserved words<br>    print(word_index.get(_ - 3))</pre><p><strong>Data Transformation</strong></p><p>Like in any other model, we have to transform our data or preprocess it, this model is not the exception. Since, neural networks work better with Tensors, we’ll have to change the dimensions of our data.</p><pre>def vectorize(sequences, dim=10000):<br>    results = np.zeros((len(sequences), dim))<br><br>    for i, sequences in enumerate(sequences):<br>        results[i, sequences] = 1<br><br>    return results</pre><p>Okay, I know this is kind of misleading, allow me to explain.</p><p>This code defines a function called vectorize that converts sequences of integers into binary matrices (tensors). Each sequence is represented as a binary vector where each element indicates the presence (1) or absence (0) of a specific integer in the sequence. The size of the binary vector is determined by the dim parameter, with default size being 10,000 because that’s the size of our data.</p><pre>X_train = vectorize(train_data)<br>x_test = vectorize(test_data)<br><br>X_train.shape<br># Our dimensions (25000, 10000)</pre><p>In the code above, we convert into tensors our train and test data. Look at the change in our dimensions.</p><p>And now, we define the target training and test labels as arrays of floats, this is often done to ensure consistent data type across the training and testing sets and to optimize memory usage.</p><pre>y_train = np.asarray(train_labels).astype(&#39;float32&#39;)<br>y_test = np.asarray(test_labels).astype(&#39;float32&#39;)</pre><p><strong>Building our model 👷🏽</strong></p><p>Now, this is what we were talking about at the beginning of this reading, the architecture of our model.</p><p>Here we initialize a sequential model which is a linear stack of layers where each layer has exactly one input tensor and one output tensor.</p><p>We use Dense layers because they work very well with text, we use two activation functions, which is what I explained at the beginning of this reading as well.</p><p>The input and hidden layer will use 16 neurons, and our output layer will use just one because our binary classification will gives us just one answer, and that’s fine because that’s all we need.</p><pre>model = models.Sequential()<br><br>model.add(layers.Dense(16, activation=&#39;relu&#39;, input_shape=(10000,)))<br>model.add(layers.Dense(16, activation=&#39;relu&#39;))<br><br>model.add(layers.Dense(1, activation=&#39;sigmoid&#39;))</pre><p>Now, we compile our model</p><pre>model.compile(optimizer=&#39;rmsprop&#39;, loss=&#39;binary_crossentropy&#39;, metrics=&#39;accuracy&#39;)</pre><p>optimizer=&#39;rmsprop&#39;: RMSprop is an optimization algorithm commonly used for training neural networks. It adapts the learning rate during training to speed up convergence. The learning rate refers to the size of the step our model takes to understand our data, and how fast it does it.</p><p>loss=&#39;binary_crossentropy&#39;: This specifies the loss function to be used during training. Binary cross-entropy is typically used for binary classification problems, where each example belongs to one of two classes.</p><p>metrics=&#39;accuracy&#39;: During training, the model&#39;s accuracy will be monitored using the accuracy metric. This metric measures the proportion of correctly classified examples.</p><p>Here we divide the data that we are going to pass to our model, this data is divided in a partial training set and a validation set</p><pre>x_val = X_train[:10000]<br>partial_x_train = X_train[10000:]<br><br>y_val = y_train[:10000]<br>partial_y_train = y_train[10000:]</pre><p>This is how we are going to validate that our model is learning.</p><p>Finally, the part I ❤️ the most! We pass the data to our model, then we tell it to do 20 different iterations or the number of times the entire training dataset will be passed forward and backward through the neural network.</p><p>Our batch_size<em> </em>defines the number of samples that will be propagated through the network at once. In this case, 512 samples will be used for each training iteration.</p><pre>history = model.fit(partial_x_train, partial_y_train,<br>                    epochs=20, batch_size=512, validation_data=(x_val, y_val))<br></pre><p>Just tell me if this doesn’t look beautiful 🥰</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zq4_6xJRtXXpJUATGzPxkg.png" /></figure><p>Let’s evaluate this model</p><pre>import matplotlib.pyplot as plt<br><br>history_dict = history.history<br>loss_values = history_dict[&#39;loss&#39;]<br>val_loss_values = history_dict[&#39;val_loss&#39;]<br><br>epoch = range(1, len(loss_values)+1)<br>plt.plot(epoch, loss_values, &#39;o&#39;, label=&#39;training&#39;)<br>plt.plot(epoch, val_loss_values, &#39;--&#39;, label=&#39;validating&#39;)<br>plt.legend()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*roLj7QKobrPejEG3Glo8pg.png" /><figcaption>Training vs validating</figcaption></figure><p>Here we can observe how our loss function goes down after every epoch, but how we’re farther from our validation data. This doesn’t look really good 🥴. Seems like our model is overfitting.</p><p>This means our model is very good at learning the patterns of the current data we’re using, but in the future if we give it some new data, the performance will be really bad.</p><p>There’s many techniques to reduce overfitting, such as <strong>Dropout</strong> and <strong>Regularization </strong>or just using less epochs. This is something we’ll be talking about a little more deeply in a next reading.</p><p><strong>In conclusion</strong>, you learned how to use Keras and TensorFlow together to build a binary classification neural network (with overfitting 😢), you learned how to vectorize your data, and the different metrics and concepts used to build one.</p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a><strong><em>.</em></strong></blockquote><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a><strong> | </strong><a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://stackademic.com/"><strong>Stackademic</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=412a32e75075" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/binary-classification-with-neural-networks-using-tensorflow-keras-412a32e75075">Binary Classification with Neural Networks using Tensorflow &amp; Keras 🧠</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simple data clustering with DBScan, Scikit-learn, and Python ✨]]></title>
            <link>https://python.plainenglish.io/simple-data-clustering-with-dbscan-scikit-learn-and-python-871d556e255e?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/871d556e255e</guid>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[dbscan]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Mon, 26 Feb 2024 22:25:27 GMT</pubDate>
            <atom:updated>2024-05-15T05:41:06.034Z</atom:updated>
            <content:encoded><![CDATA[<h3>Simple Data Clustering with DBScan, Scikit-learn, and Python 🔍✨</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0osNtTq581HV9CBP" /><figcaption>Star cluster</figcaption></figure><p>Clustering is a fundamental technique employed in data science to identify inherent patterns and structures within datasets without the need for labeled data. Unlike supervised learning methods, where the algorithm is trained on labeled examples to make predictions about a specific target variable, unsupervised learning, including clustering, involves discovering the natural groupings or clusters that exist within the data.</p><p>In clustering, the algorithm automatically partitions the dataset into groups based on the similarity of observations or data points, with the objective of maximizing intra-cluster similarity and minimizing inter-cluster similarity. This process allows for the identification of similarities and differences among data points, facilitating tasks such as customer segmentation, anomaly detection, and pattern recognition.</p><p>Common clustering algorithms include K-means, hierarchical clustering, and density-based clustering methods like DBSCAN. Each algorithm has its own strengths and weaknesses, making them suitable for different types of datasets and clustering objectives.</p><p>For example, we could use clustering algorithms to segment countries based on a set of features (GDP, child mortality, education, population) given to each country.</p><p>In this opportunity, we are going to be talking about DBScan. I love this particular model because it is pretty intuitive.</p><p><strong>Getting into it</strong></p><p>We’ll have to import the following dependencies, make sure you have them installed.</p><pre>import numpy as np<br>import pandas as pd<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br>from sklearn.datasets import make_blobs, make_moons<br>from sklearn.cluster import DBSCAN<br>from sklearn.metrics import silhouette_score</pre><p>For this specific case, we are going to use synthetic data.</p><p>What we’re doing in the code below is creating 500 data points, our data is going to be clustered around 4 different centers, 0.8 is our standard deviation (how spread out is our data), 2 are going to be the number of dimensions in our data, and finally we’re specifying that the centers of the clusters should also be returned along with the data points. The centers are stored in the variable c.</p><pre>X, y, c = make_blobs(500, centers=4, cluster_std=0.8, n_features=2, return_centers=True)</pre><p>Here we assign the data points and center with the coordinates in ‘x1’ and ‘x2’ since it’s two-dimensional</p><pre>df_blobs = pd.DataFrame({<br>    &#39;x1&#39;: X[:,0],<br>    &#39;x2&#39;: X[:,1],<br>})<br><br>df_centers = pd.DataFrame({<br>    &#39;x1&#39;: c[:,0],<br>    &#39;x2&#39;: c[:,1],<br>})</pre><p>Now, if we visualize our data:</p><pre>sns.scatterplot(data=df_blobs, x=&#39;x1&#39;, y=&#39;x2&#39;)<br>sns.scatterplot(data=df_centers, x=&#39;x1&#39;, y=&#39;x2&#39;, marker=&#39;X&#39;, s=100)<br>plt.plot()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*42l7PNPjobe0nfNEQnIV8w.png" /><figcaption>Data blobs and data centers adjusted</figcaption></figure><p>We can see how we have an ‘X’ mark in each one of our data clusters, but this is something which was already fixed, we want an algorithm to learn and find these separations in the data for us.</p><p>This why we are going to use DBScan. With DBScan, we set an epsilon which makes reference to the radius of the circle, and a minimum number of points, which would mean a minimum number of clusters,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6YnkSao0ETNj0_502Jw74A.gif" /><figcaption><a href="https://www.naftaliharris.com/blog/visualizing-dbscan-clustering/">https://www.naftaliharris.com/blog/visualizing-dbscan-clustering/</a></figcaption></figure><p>We obtain this as the final result for this small experiment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d0GUKfB4s4Y8fV4wE-BqNg.png" /></figure><p><strong>Implementing DBScan</strong></p><p>Continuing with the data we already had created, let’s implement DBScan:</p><pre>db_scan = DBSCAN(eps=1.5, min_samples=4)<br>y_predict = db_scan.fit_predict(X)<br>df_blobs[&#39;cluster&#39;] = y_predict<br><br>sns.scatterplot(data=df_blobs, x=&#39;x1&#39;, y=&#39;x2&#39;, hue=&#39;cluster&#39;)<br>plt.plot()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*QL_pBO4og0ewLuSOp-ftKQ.png" /><figcaption>DBScan Clusters</figcaption></figure><p>We obtained 4 clusters, but if we make our eps (epsilon) smaller, we could obtain more clusters.</p><p>Let’s prove this with hyperparameters.</p><pre>from sklearn.neighbors import NearestNeighbors<br><br>neighbors = NearestNeighbors(n_neighbors=3)<br>neighbors_fit = neighbors.fit(X)<br>distances, indexes = neighbors_fit.kneighbors(X)<br><br>distances = np.sort(distances, axis=0)<br>distances = distances[:,1]<br>plt.plot(distances)</pre><p>In the code above, we use the Kneedle algorithm to detect the knee with clustering so we can define the optimum number of clusters to create.</p><p>In this case, we see that the knee is between 0.2 and 0.5.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*jZEuzKwVgkAjldEsppykSQ.png" /><figcaption>Knee point detection</figcaption></figure><p>Here we define a range for the epsilon values, it’ll go from 0.25 to 0.5, increasing by 0.10 in each iteration. For the minimum samples, we’ll go from 2 to 10.</p><pre>eps_values = np.arange(0.25, 0.5, 0.10)<br>min_samples = np.arange(2, 10)</pre><p>Here, we iterate through these values and we finally obtain the silhouette score for each prediction.</p><pre>from itertools import product<br><br>dbscan_params = list(product(eps_values, min_samples))<br>sil_scores = []<br><br>for p in dbscan_params: <br>    y_pred = DBSCAN(eps=p[0], min_samples=p[1]).fit_predict(X)<br>    sil_scores.append(silhouette_score(X, y_pred))</pre><p>Here, we take the new params and the silhouette scores, put them into a matrix which makes it way easier to visualize.</p><pre>df_params_tuning = pd.DataFrame.from_records(dbscan_params, columns=[&#39;Eps&#39;, &#39;Min_samples&#39;])<br>df_params_tuning[&#39;sil_scores&#39;] = sil_scores<br><br>pivot_data = pd.pivot_table(df_params_tuning, values=&#39;sil_scores&#39;, index=&#39;Min_samples&#39;, columns=&#39;Eps&#39;)<br><br>plt.subplots(figsize=(18,6))<br>sns.heatmap(pivot_data, annot=True, annot_kws={&#39;size&#39;: 10}, cmap=&#39;coolwarm&#39;)<br>plt.show()</pre><p>Here we can observe the different silhouette scores for each min_sample and epsilon.</p><p>The silhouette score shows how well an item fits within its own cluster compared to others. It ranges from -1 to +1, with higher scores meaning better fit within its cluster and poorer fit with neighboring clusters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qm3T83t83E7jjSejOQgO3Q.png" /></figure><p><strong>Evaluating DBScan</strong></p><p>Using the logic established before, we will use an eps of 0.5 with a min_samples value of 4. We run the following code.</p><pre>dbscan_cluster = DBSCAN(eps=0.5, min_samples=4)<br>y_predict = dbscan_cluster.fit_predict(X)<br>df_blobs[&#39;cluster&#39;] = y_predict<br><br>sns.scatterplot(data=df_blobs, x=&#39;x1&#39;, y=&#39;x2&#39;, hue=&#39;cluster&#39;)<br>plt.plot()<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*3mgmothKObuzamtLVsae4w.png" /><figcaption>DBScan Clustering</figcaption></figure><p>Here we can verify what we said before about how using a smaller epsilon could get us more clusters, and finally obtain the following silhouette score</p><pre>clusterer = DBSCAN(eps=0.7, min_samples=3)<br>cluster_labels = clusterer.fit_predict(X)<br>n_clusters = len(np.unique(cluster_labels))<br><br>silhouette_avg = silhouette_score(X, cluster_labels)<br>print(<br>    &quot;n_clusters =&quot;,<br>    n_clusters,<br>    &quot;The average silhouette_score is :&quot;,<br>    silhouette_avg,<br>)</pre><p>In my case, it was: <em>n_clusters = 5 The average silhouette_score is : 0.7413759260585894</em></p><p><strong>In conclusion, </strong>our DBScan model obtained a great silhouette score, and you learned how to implement DBScan in your clustering projects and evaluate it, and how to find better results using the Kneedle algorithm and hyperparameters.</p><p>More models here: <a href="https://github.com/danhergir/clustering">https://github.com/danhergir/clustering</a></p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a></blockquote><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a><strong> | </strong><a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://stackademic.com/"><strong>Stackademic</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=871d556e255e" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/simple-data-clustering-with-dbscan-scikit-learn-and-python-871d556e255e">Simple data clustering with DBScan, Scikit-learn, and Python 🔍✨</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Income Classification with Random Forests and Scikit-learn]]></title>
            <link>https://danhergir.medium.com/income-classification-with-random-forests-and-scikit-learn-cdbf688548ed?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdbf688548ed</guid>
            <category><![CDATA[random-forest]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Sat, 17 Feb 2024 06:24:47 GMT</pubDate>
            <atom:updated>2024-02-17T06:29:21.800Z</atom:updated>
            <content:encoded><![CDATA[<h3>Income Prediction with Random Forests and Scikit-learn🌳</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Jb50CvEONfPCOXL4" /><figcaption>Forest but no Gump</figcaption></figure><p>Decision trees are a popular and intuitive tool used in machine learning and data mining for classification and regression tasks. They represent a flowchart-like structure where each internal node represents a “test” on an attribute, each branch represents the outcome of the test, and each leaf node represents a class label or a numerical value. They are called “trees” because they start with a single node, which then splits into branches, and these branches can further split into more branches, forming a tree-like structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jNUrctQqcLhxht1c.png" /><figcaption>IBM | <a href="https://www.ibm.com/topics/decision-trees">https://www.ibm.com/topics/decision-trees</a></figcaption></figure><p>Now, Random Forests? It’s like having a bunch of decision trees working together on the same problem. Each tree gets its own slice of the data to train on, and then they all come together to vote on the best solution. It’s like teamwork for models! And yeah, it’s super handy because it helps prevent overfitting and deals well with noisy data. Plus, it’s quick since each tree can be trained independently. Overall, it’s a cool method for both classification and regression tasks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*Xz0tdxbXsLu9cAat" /><figcaption>IBM | <a href="https://www.ibm.com/topics/random-forest">https://www.ibm.com/topics/random-forest</a></figcaption></figure><p><strong>Project Introduction</strong></p><p>In this opportunity, we are going to be using Random Forests to do some classification using the following dataset: <a href="https://archive.ics.uci.edu/dataset/20/census+income">Census+Income</a></p><p>We’re going to predict whether income exceeds $50K/yr or not, and play with the features which contribute the most to our model.</p><p>Let’s start by importing the data.</p><pre>import pandas as pd<br>import matplotlib.pyplot as plt<br>import seaborn as sns<br><br>df1 = pd.read_table(&#39;./data/income/adult.data&#39;, delimiter=&#39;,&#39;, header=None)<br><br>df2 = pd.read_table(&#39;./data/income/adult.test&#39;, delimiter=&#39;,&#39;, header=None)<br><br># Merge train and test dataframes<br>df = pd.concat([df1, df2])</pre><p>I know that both datasets are already split, but this is something I did on purpose to play with randomness, also if you explore the dataset you’re going to find a lot of null values, and this is something we’d like to avoid, so our model can perform better.</p><p>Let’s change the name of our columns.</p><pre>df.columns = [&#39;age&#39;, &#39;workclass&#39;, &#39;fnlwgt&#39;, &#39;education&#39;, &#39;education-num&#39;, &#39;marital-status&#39;, &#39;occupation&#39;, &#39;relationship&#39;, &#39;race&#39;, &#39;sex&#39;, &#39;capital-gain&#39;, &#39;capital-loss&#39;, &#39;hours-per-week&#39;, &#39;native-country&#39;, &#39;target&#39;]</pre><p>I found out that null values in this dataset actually can be found with an ‘ <em>?’</em> sign.</p><pre>contains_question_mark = df.applymap(lambda x: &#39; ?&#39; in str(x)).any().any()<br><br>print(&quot;Does the DataFrame contain &#39;?&#39; anywhere?&quot;, contains_question_mark)<br><br># If yes, count the occurrences<br>if contains_question_mark:<br>    count_question_marks = (df.applymap(lambda x: x == &#39; ?&#39;)).sum().sum()<br>    print(&quot;Number of occurrences of &#39; ?&#39;:&quot;, count_question_marks)</pre><p>Here’s how the null values are distributed in the different columns.</p><pre>df = df.replace(&#39; ?&#39;, pd.NA)<br>df.isnull().sum()</pre><p>Now we get rid of the null values. <em>This is something I don’t suggest to do all the time, but just this time it won’t really affect the performance of our model</em></p><pre>df.dropna(inplace=True)</pre><p>Now, we’re going to find the unique values of our target column.</p><pre>df_new = pd.get_dummies(df, columns=[&#39;target&#39;], drop_first=True)<br>df.target.unique()</pre><p>We see we have the following unique values:</p><p><strong>‘ &lt;=50K’, ‘ &gt;50K’, ‘ &lt;=50K.’, ‘ &gt;50K.’</strong></p><p>This is something we want to standardize, so instead of working with categorical values, we can work with numerical ones.</p><pre>def modify_value(value):<br>    if value == &#39; &lt;=50K&#39; or value == &#39; &lt;=50K.&#39;:<br>        return 0<br>    else:<br>        return 1<br><br># Apply the function to the column using apply()<br>df[&#39;target&#39;] = df[&#39;target&#39;].apply(modify_value)</pre><p><strong>Train &amp; Test split data</strong></p><p>I excluded these specific columns due to their continuous nature, which could introduce overfitting into our model. Our test data is going to have 30% of our total data.</p><pre>X = df.drop([&#39;target&#39;, &#39;fnlwgt&#39;, &#39;age&#39;, &#39;capital-gain&#39;, &#39;capital-loss&#39;], axis=1)<br>y = df.target<br><br>from sklearn.model_selection import train_test_split<br><br>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3, random_state=00000)</pre><p><strong>Encoding our columns</strong></p><p>This code uses the OrdinalEncoder from the category_encoders library to convert categorical variables into numerical values. It specifies certain columns to encode, fits the encoder to the training data, and then transforms both the training and test data accordingly.</p><p>It’s necessary to perform this encoding to ensure that the categorical variables are compatible with machine learning algorithms, which typically operate on numerical data</p><pre>import category_encoders as ce<br><br>encoder = ce.OrdinalEncoder(cols = [&#39;workclass&#39;, &#39;education&#39;, &#39;marital-status&#39;, &#39;occupation&#39;,&#39;relationship&#39;, &#39;race&#39;, &#39;sex&#39;, &#39;native-country&#39;])<br><br>X_train = encoder.fit_transform(X_train)<br><br>X_test = encoder.transform(X_test)</pre><p><strong>Training model</strong></p><p>Here <em>n_estimators</em> makes reference to the number of Decision Trees that we’re going to use on this model</p><pre>from sklearn.ensemble import RandomForestClassifier<br><br>random_forest = RandomForestClassifier(n_estimators=20, random_state=00000)<br><br>random_forest.fit(X_train, y_train)</pre><p><strong>Evaluating our model</strong></p><p>Here we try to obtain the accuracy score for our train data and for our test data.</p><blockquote>The <strong>accuracy score</strong> represents the proportion of correctly classified instances out of the total instances in a dataset. It is calculated by dividing the number of correct predictions by the total number of predictions made. In classification tasks, accuracy is a commonly used metric to evaluate the performance of a model, indicating how well it predicts the correct class labels.</blockquote><pre>from sklearn.metrics import accuracy_score<br><br>train_accuracy_rf = accuracy_score(y_train, y_train_pred_rf)<br><br>test_accuracy_rf = accuracy_score(y_test, y_test_pred_rf)<br><br>print(train_accuracy_rf, test_accuracy_rf)</pre><p><strong>Train: 0.8946138050860843 / Test: 0.8180880076656594</strong></p><p>The training accuracy of 0.8946 suggests that the model is able to correctly classify approximately 89.46% of the instances in the training dataset. This indicates that the model performs relatively well on the data it was trained on.</p><p>The test accuracy of 0.8181 indicates that the model can correctly classify about 81.81% of the instances in the unseen test dataset. While this is lower than the training accuracy, it still shows that the model generalizes reasonably well to new, unseen data.</p><p><strong>Features Performance</strong></p><p>This chart shows us the features which contributed the most to our model. Something I would suggest based on these results is to go back and play with the features we dropped back when we were splitting our dataset, and see how these changes the performance of our model.</p><p>The correlation between the number of hours worked per week and financial prosperity is apparent. Similarly, the influence of education on income seems intuitive. Additionally, the impact of relationships and marital status on financial success is evident, as commitments and responsibilities often necessitate increased effort and earnings.</p><pre>import seaborn as sns<br>import matplotlib.pyplot as plt<br><br>importances = random_forest.feature_importances_<br>columns = X.columns<br><br>sns.barplot(x=importances , y = columns)<br>plt.xlabel(&#39;Features Importance Score&#39;)<br>plt.ylabel(&#39;Feature&#39;)<br>plt.title(&quot;Visualizing Features Importances&quot;)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*PYj_Ez27E9NSsVhrcTf-VQ.png" /><figcaption>Features Performance</figcaption></figure><p><strong>Finally, our classification report:</strong></p><pre>from sklearn.metrics import classification_report<br><br>print(classification_report(y_test,y_test_pred_rf))</pre><pre>              precision    recall  f1-score   support<br><br>           0       0.86      0.90      0.88     10258<br>           1       0.65      0.55      0.60      3309<br><br>    accuracy                           0.82     13567<br>   macro avg       0.76      0.73      0.74     13567<br>weighted avg       0.81      0.82      0.81     13567</pre><ul><li>Recall: It captured 90% of class 0 instances and 55% of class 1 instances.</li><li>F1-score: A balance of precision and recall, with higher scores indicating better performance. Class 0 has an F1-score of 0.88, while class 1 has 0.60.</li><li>Accuracy: Overall correctness of the model, which is 82%.</li><li>Support: Number of instances for each class.</li><li>Macro avg: Average precision, recall, and F1-score across classes.</li><li>Weighted avg: Average precision, recall, and F1-score considering class imbalance.</li></ul><p>In summary, the model performs better at classifying class 0 but struggles with class 1 due to lower precision, recall, and F1-score.</p><p><strong>In conclusion, </strong>our model could be better, but that’s something you can improve.</p><p>I’d ❤️ to see you to play with this by dropping or adding new columns.</p><p>In this opportunity you learned about Decision Trees and Random Forests with Scikit-learn, and you were able to obtain the features that would make your model perform better at classifying people that can make more or less than $50K.</p><p>More models here: <a href="https://github.com/danhergir/decision-trees">https://github.com/danhergir/decision-trees</a></p><p>More info about this project: <a href="https://cseweb.ucsd.edu/classes/sp15/cse190-c/reports/sp15/048.pdf">https://cseweb.ucsd.edu/classes/sp15/cse190-c/reports/sp15/048.pdf</a></p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdbf688548ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Multi-class Logistic Regression with Scikit-learn ]]></title>
            <link>https://danhergir.medium.com/implementing-multi-class-logistic-regression-with-scikit-learn-53d919b72c13?source=rss-190dfb810abc------2</link>
            <guid isPermaLink="false">https://medium.com/p/53d919b72c13</guid>
            <category><![CDATA[multiclass-classification]]></category>
            <category><![CDATA[multi-class]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[scikit-learn]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <dc:creator><![CDATA[Daniel Hernandez]]></dc:creator>
            <pubDate>Sat, 10 Feb 2024 04:46:18 GMT</pubDate>
            <atom:updated>2024-02-10T04:46:18.222Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Hj7aU0KUj5NOT4Vg" /><figcaption>Beans (Multiple Classes)</figcaption></figure><p>Normally we use logistic regression to classify elements in a binary manner, meaning that a prediction is either true or false. But using this same method, we can also perform classification of multiple classes or values.</p><p>To perform logistic regressions, we rely on the sigmoid function, which allows us to establish values between 0 and 1 in a single category. In contrast, multinomial logistic regression uses the softmax function. Softmax assigns probabilities to each category such that the sum of the probabilities for all categories equals one. This ensures that the predicted probabilities represent a valid probability distribution.</p><p><strong>Let’s get into it! </strong>✨</p><p>You can download the dataset from <a href="https://www.kaggle.com/datasets/sansuthi/dry-bean-dataset/code">here</a></p><p>Here are some the requirements you’ll need to install to start this project:</p><pre>pip install matplotlib pandas scikit-learn imblearn</pre><p>Let’s import some dependencies and take a look at our data</p><pre>import matplotlib.pyplot as plt<br>import numpy as numpy<br>import pandas as pd<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.metrics import confusion_matrix, accuracy_score, ConfusionMatrixDisplay<br>from sklearn.preprocessing import StandardScaler<br>import seaborn as sns<br>sns.set()<br><br>df = pd.read_csv(&#39;./data/Dry_Bean.csv&#39;)<br>df[&#39;Class&#39;].unique()</pre><p>The “Class” column refers to the categories of dry beans in our dataset which are: Seker, Barbunya, Bombay, Cali, Horoz, Sira, Dermason.</p><p>We can appreciate that our dataset does not have null or na values which is going to make our exploratory analysis way easier. You can check this by running the following line.</p><pre>df.isnull().sum()</pre><p><strong>Exploring categories and resampling 🚀</strong></p><p>We’re going to check the number of values we have for our different classes</p><pre>sns.countplot(x=&#39;Class&#39;, data=df)<br>plt.xticks(rotation=45)<br>plt.show()</pre><p>First thing we can appreciate out of this chart is the lack of values for some classes, in order to properly train our model it would be nice to have a more balanced distribution in our dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*iZR-4soXr24UakaVfpQ9Lw.png" /><figcaption>Bean Classes Count</figcaption></figure><p>In the piece of code below, we are using a technique called random under-sampling, it involves randomly selecting examples from the majority class and deleting them from the training dataset. In the random under-sampling, the majority class instances are discarded at random until a more balanced distribution is reached</p><pre>import imblearn<br>from imblearn.under_sampling import RandomUnderSampler<br><br>undersample = RandomUnderSampler(random_state=42)<br><br>X = df.drop(&#39;Class&#39;, axis=1)<br>y = df.Class<br><br>X_over, y_over = undersample.fit_resample(X, y)<br><br>sns.countplot(x=y_over, data=df)<br>plt.xticks(rotation=45)<br>plt.show()</pre><p>This how a more balanced distribution looks like 👀</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/575/1*pimi8h2oDz9BtjEtPd2PiA.png" /><figcaption>Under-sampling class distribution</figcaption></figure><p>If you run the following code, you’ll see that we ended up with less rows in our dataset.</p><pre># This is the shape of our dataset before balancing<br>df.shape<br><br># Shape of dataset after balance<br>X_over.shape</pre><p>Now, let’s focus on doing the numerical conversion of our categories, and give a reference number to every class.</p><pre>import numpy as np<br>y_over.replace(list(np.unique(y_over)), [1, 2, 3, 4, 5, 6, 7], inplace=True)</pre><p>Once we’ve done this, we can move into finding correlations between our data columns.</p><pre>df_dea = X_over<br>df_dea[&#39;Class&#39;] = y_over<br><br>plt.figure(figsize=(15, 10))<br>sns.heatmap(df_dea.corr(), annot=True)<br>plt.show()</pre><p>Here we can observe how there’s a high correlation between the columns ‘convexArea’ and ‘EquivDiameter’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e9ZHQbnlq31UexcGUtJT4Q.png" /><figcaption>Correlation Heatmap</figcaption></figure><p>The high correlation between those two columns could make us create an overfitted model, and that’s something we really want to avoid. Let’s get rid out of those columns.</p><pre># This columns may create an overfitted model<br>X_over.drop([&#39;ConvexArea&#39;, &#39;EquivDiameter&#39;], axis=1, inplace=True)</pre><p><strong>Building our model 🦾</strong></p><p>Now, let’s get down to business! We are going to start training our model!</p><p>In order to do that, we will start by splitting our train and test data, we’re going to be using the 20% of our dataset for our test data.</p><pre>X_train, X_test, y_train, y_test = train_test_split(X_over, y_over, random_state=42, shuffle=True, test_size=.2)</pre><p>Obviously, we can’t forget to scale our data. This is important because it help us to balance the impact of all variables on the distance calculation and can help to improve the performance of the algorithm,</p><pre>st_x = StandardScaler()<br>X_train = st_x.fit_transform(X_train)<br>X_test = st_x.transform(X_test)</pre><p>For our Multi-class model, we are going to try several solvers and multiclass algorithms, that’s why we are going to use the following function.</p><pre>def logistic_model(C, solver_, multiclass_):<br>    logistic_regression_model = LogisticRegression(random_state=42, solver=solver_, multi_class=multiclass_, n_jobs=1, C=C)<br>    return logistic_regression_model</pre><p>And we’re going to pass it the following arguments, obtain the accuracy scores, and then display the results.</p><pre>multiclass = [&#39;ovr&#39;, &#39;multinomial&#39;]<br>solver_list = [&#39;liblinear&#39;, &#39;newton-cg&#39;, &#39;lbfgs&#39;, &#39;sag&#39;, &#39;saga&#39;]<br>scores = []<br>params = []<br><br>for i in multiclass:<br>    for j in solver_list:<br>        try:<br>            model = logistic_model(1, j, i)<br>            model.fit(X_train, y_train)<br>            predictions = model.predict(X_test)<br>            params.append(i + &#39; &#39; + j)<br>            accuracy = accuracy_score(y_test, predictions)<br>            scores.append(accuracy)<br>        except:<br>            None<br><br>sns.barplot(x=params, y=scores).set_title(&#39;Beans Accuracy&#39;)<br>plt.xticks(rotation=90)</pre><p>We can appreciate that all of our models had an amazing performance, but we’ll have to stick to the ‘multinomial newton-cg’ which is the one with the highest accuracy score.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*X0EdNB7frwW18Ap3oC501Q.png" /><figcaption>Accuracy scores</figcaption></figure><p>Here we can appreciate a little bit better the results for our model:</p><pre>model = logistic_model(1, &#39;newton-cg&#39;, &#39;multinomial&#39;)<br>model.fit(X_train, y_train)<br>predictions = model.predict(X_test)<br>print(confusion_matrix(y_test, predictions))<br>print(accuracy_score(y_test, predictions))<br><br><br># Confusion Matrix Heatmap<br>cm = confusion_matrix(y_test, predictions, labels=model.classes_)<br>disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=model.classes_)<br>disp.plot()<br>plt.show()</pre><pre>[[128   0   0   0   0   0   0]<br> [  0  98   0   0   0   0   0]<br> [  0   0 102   1   0   0   0]<br> [  0   0   0 105   0   0   0]<br> [  0   0   1   1  94   0   0]<br> [  0   0   0   0   0  90   1]<br> [  0   0   0   0   0   0 110]]<br><br>Prediction: 0.9945280437756497</pre><p>Here are the results for our ‘multinomial newton-cg’. Curious how our model made only just 4 mistakes 🎯.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*U45nPoK1UyRgaoDOLn4-9Q.png" /><figcaption>Confusion Matrix Heat map</figcaption></figure><p><strong>Conclusion:</strong></p><p>If you’d like to explore this code a little bit more or take a look at other Logistic Regression projects, visit this <a href="https://github.com/danhergir/Logistic_regression">repository</a>. I’d ❤️ to see any kind of PR or feedback.</p><p>In this reading, you learned how to use Multi-class Logistic Regression, how to scale your data, and why to use random under-sampling.</p><blockquote><em>To learn more about me and discover additional insights </em><strong><em>visit </em></strong><a href="http://danhergir.com/"><strong><em>https://danhergir.com</em></strong></a><em>. You can also explore my Medium articles by </em><strong><em>visiting </em></strong><a href="https://danhergir.medium.com/"><strong><em>https://danhergir.medium.com</em></strong></a><strong><em> </em></strong><em>for more in-depth content or connect with me on Twitter @ </em><a href="http://twitter.com/danhergir"><strong><em>https://twitter.com/danhergir</em></strong></a></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53d919b72c13" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>