<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ansuman Bhujabala on Medium]]></title>
        <description><![CDATA[Stories by Ansuman Bhujabala on Medium]]></description>
        <link>https://medium.com/@ansumanbhujabal?source=rss-3c61f3cddec------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Zvss3lqTWJz7zneaV49hsw.jpeg</url>
            <title>Stories by Ansuman Bhujabala on Medium</title>
            <link>https://medium.com/@ansumanbhujabal?source=rss-3c61f3cddec------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 15:56:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ansumanbhujabal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Most basic RAG implementation for begginers (interviewer asked for GenAI Intern role )]]></title>
            <link>https://ansumanbhujabal.medium.com/most-basic-rag-implementation-for-begginers-interviewer-asked-for-genai-intern-role-6a864fecc533?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a864fecc533</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[interview]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Mon, 13 Jan 2025 16:21:49 GMT</pubDate>
            <atom:updated>2025-01-13T16:21:49.177Z</atom:updated>
            <content:encoded><![CDATA[<p>Today I will share the implementation problem I was asked to do for a GenAI intern role at a Data centric startup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gCURKKy8FtkkSSg1" /><figcaption><a href="https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2023/11/llama-rag.jpg?resize=768%2C480&amp;ssl=1">RAG with Llama</a></figcaption></figure><p><strong>Scenario-</strong></p><p><em>30 Minutes time , Machine round with full screen share an llowed access to Google , restriction on -LLM</em></p><p><strong>Question-</strong> <strong>Perform RAG on your own resume and show what the LLM thinks.</strong></p><p><strong>My approach-</strong></p><p>Explained the overall approach and what techstack I’ll be using for it to the interviewer in first 2 minutes.</p><p>I’ll be adding the full code with requirements and all possible questioning area with answers in comments .</p><p><strong>Techstack: </strong>Python3, Llama, Groq, Pinecone , LightningAi(platform)</p><p><strong>Code :</strong></p><pre>import os  ## used to read file path <br>import time<br>from dotenv import load_dotenv ## to load api keys from env<br>from langchain_community.document_loaders.pdf import PyPDFDirectoryLoader,PyPDFLoader ## To load and read pdf files <br>from langchain_text_splitters.character import CharacterTextSplitter, RecursiveCharacterTextSplitter ## To split the documents in chunks <br>from langchain_community.embeddings import HuggingFaceEmbeddings ## To  vectorize the chunks <br>from pinecone import Pinecone,ServerlessSpec ## To connect with Pinecone <br>from langchain_pinecone import PineconeVectorStore ## To store indexes in PineconeVectorStore<br>from langchain_groq import ChatGroq ## To load LLM<br>from langchain.chains.question_answering import load_qa_chain ## To make the question answering chain<br>import warnings<br>warnings.filterwarnings(&#39;ignore&#39;) ## To ignore all warnings coming in the execution<br><br>load_dotenv() ## Load all env<br><br><br><br>## Load api keys <br>api_key=os.getenv(&quot;PINECONE_API_KEY&quot;)<br>if not api_key:<br>    raise ValueError(&quot;PINECONE_API_KEY is not found&quot;)<br><br>## Initialize Pinecone <br>pc=Pinecone(api_key=api_key)<br>index_name=&quot;practice&quot;  ## Only lowercase allowed in pinecone index<br><br>## Initialize LLM<br><br>llm= ChatGroq(<br>    model=&quot;llama-3.1-70b-versatile&quot;,<br>    temperature=0<br>)<br>## Initialize Chain with LLM<br>chain=load_qa_chain(llm, chain_type=&quot;stuff&quot;)<br><br>## Load docs <br>def read_doc(directory):<br>    base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))<br>    print(f&quot;The base dir is {base_dir}&quot;)<br>    full_path = os.path.join(base_dir, directory)<br>    print(f&quot;The path dir is {full_path}&quot;)    <br>    if not os.path.exists(full_path):<br>        raise FileNotFoundError(f&quot;Directory &#39;{directory}&#39; does not exist.&quot;)<br>    file_loader=PyPDFDirectoryLoader(full_path)<br>    print(file_loader)<br>    documents=file_loader.load()  ## Load the files from the path <br>    return documents<br><br>## Chuck documents <br>def chunk_data(docs, chunk_size=800, chunk_overlap=80):## Chunk overlap to keep the document parts consistent in every chunk <br>    text_splitter=RecursiveCharacterTextSplitter(chunk_size=chunk_size,chunk_overlap=chunk_overlap) ## Recursively chuk the docs w.r.t the overlap<br>    chunked_docs=text_splitter.split_documents(docs)<br>    return chunked_docs<br><br># ## Use the 2 functions<br>doc=read_doc(&quot;upload&quot;)<br><br># print(f&quot;The doc is {doc}&quot;)<br>chunked_docs=chunk_data(doc)<br>print(f&quot;The chunked doc is {len(chunked_docs)}&quot;)<br><br><br>## Embeddings <br>embeddings = HuggingFaceEmbeddings()<br>embeddings_dimension=768 ## Default weight by Huggingface to avoid any troubles <br>print(f&quot;The dimension of embeddings is {embeddings_dimension}&quot;)<br><br>## Check if Index is already in pinecone otherwise create one <br><br>if index_name not in [index_info[&quot;name&quot;] for index_info in pc.list_indexes()]:<br>    pc.create_index(<br>        name=index_name,<br>        dimension=embeddings_dimension,<br>        metric=&#39;cosine&#39;,<br>        spec=ServerlessSpec(cloud=&#39;aws&#39;,region=&#39;us-east-1&#39;)<br>    )<br>    ## Insert document embeddings into the index <br>    index=PineconeVectorStore.from_documents(<br>        documents= chunked_docs,<br>        index_name=index_name,<br>        embedding=embeddings<br>    )<br>    pc.describe_index()<br>    ## Wait for some seconds to connect <br>    while not pc.describe_index(index_name).staus[&quot;ready&quot;]:<br>        time.sleep(1)<br><br>## Connect to index <br>index=pc.Index(index_name)<br>print(f&quot; The index is connected successfully {index}&quot;) <br><br>## Load the vectors from index <br>index = PineconeVectorStore.from_existing_index(<br>    index_name=index_name,<br>    embedding=embeddings<br>)<br>## Retrieve matching documents<br><br>def retrieve_query(query,k=2):    ## K is the number of top matching documents to consider <br>    matching_result=index.similarity_search(query=query,k=k)  ## Based on cosine similarity of document and query<br>    return matching_result<br><br>## Function to get an answer<br><br>def retrieve_answer(query):<br>    ## Get the matching indexes<br>    doc_search=retrieve_query(query)<br>    ## Invoke the chain <br>    answer=chain.run(input_documents=doc_search, question=query)<br>    return answer<br><br>if __name__==&quot;__main__&quot;:<br>    ans=retrieve_answer(&quot;Tell me what you think about the person  ? &quot;)<br>    print(ans)<br><br></pre><p><strong>Sample Output :</strong></p><pre>Based on the provided context, it appears that the person is:<br><br>1. Tech-savvy: They have experience with natural language processing, machine learning, and data pipelines, indicating a strong technical background.<br>2. Project-oriented: They have worked on multiple projects, such as BudgetExpert RAG, Paraphrasing-Engine, and Mindflow, which suggests that they are proactive and enjoy taking on new challenges.<br>3. Communicative: They write about their project findings on Medium and have a significant readership, which implies that they are able to effectively communicate their ideas and thoughts.<br>4. Community-oriented: They are an admin for a technology community, which suggests that they are interested in collaborating with others and contributing to the growth of their community.<br>5. Possibly a student or young professional: Their participation in Major League Hacking and their role as an admin for a technology community comprising individuals from various colleges implies that they may be a student or a young professional.<br><br>Overall, the person seems to be a motivated, technically skilled, and community-oriented individual with a strong interest in natural language processing and machine learning. However, it&#39;s essential to note that this is a limited perspective based on a brief context, and a more comprehensive understanding of the person would require more information.</pre><p><strong>requirements.txt : ( I have some not used here too)</strong></p><pre>fastapi<br>uvicorn<br>PyMuPDF<br>langchain<br>llama-index<br>pinecone-client<br>sqlalchemy<br>pydantic<br>requests<br>boto3<br>python-dotenv==1.0.1<br>streamlit==1.37.0<br>langchain-community==0.2.10<br>langchain-text-splitters==0.2.2<br>langchain-groq==0.1.9<br>unstructured==0.15.0<br>unstructured[pdf]==0.15.0<br>faiss-gpu==1.7.2<br>transformers==4.43.3<br>sentence-transformers==3.0.1<br>nltk==3.8.1</pre><p><strong>Follw-up questions asked</strong> ( all answers are in commented part):</p><ol><li>Why Chunk overlap and how it helps?</li><li>Why embeddings_dimension is 768?</li><li>What is “K” ? How it affects the time ?</li><li>What is “stuffed chain “ ?</li></ol><blockquote><em>A “stuff chain” specifically refers to a simple chain type where the input documents are “stuffed” into a single prompt and then passed to the language model for processing. This is useful for tasks where you want to combine multiple documents into a single context before generating a response.</em></blockquote><p><strong>Problems and tricky situations :</strong></p><p>1.</p><pre>from dotenv import load_dotenv ## to load api keys from env<br>from langchain_community.document_loaders.pdf import PyPDFDirectoryLoader,PyPDFLoader ## To load and read pdf files <br>from langchain_text_splitters.character import CharacterTextSplitter, RecursiveCharacterTextSplitter ## To split the documents in chunks <br>from langchain_community.embeddings import HuggingFaceEmbeddings ## To  vectorize the chunks <br>from pinecone import Pinecone,ServerlessSpec ## To connect with Pinecone <br>from langchain_pinecone import PineconeVectorStore ## To store indexes in PineconeVectorStore<br>from langchain_groq import ChatGroq ## To load LLM<br>from langchain.chains.question_answering import load_qa_chain ## To make the question answering chain</pre><p>See , Langchain has a lot of global , community , core and specific dependencies and functions , and all can be used to solve the above problem .</p><p>But as the time is a constraint here , experimenting and testing will not be in our favor .</p><p>So a little familiarity and practice can save a lot of time .</p><p>2. <strong>Pinecone:</strong></p><p>Pinecone can give error for the simplest change in libaray import and their docs gets update frequently , so it can be a problem too.</p><p>Sticking to langchain_pinecone for PineconeVectorStore won’t make any VectorStore related issue .</p><p>Hope this will help you in anyway.</p><p><strong>Connect with Me on GitHub and LinkedIn</strong></p><p>If you found this journey through RAG as fascinating as I did, let’s connect! You can explore the code and examples discussed in this blog on my GitHub repository. Also, if you’re interested in more devops-related insights and cloud adventures, follow me on LinkedIn.</p><p><a href="https://github.com/Ansumanbhujabal"><strong>GitHub </strong></a><strong>| </strong><a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/"><strong>LinkedIn</strong></a></p><p>Let’s dive deeper into the data-driven world together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a864fecc533" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Named Volumes What , When & How for Begginers]]></title>
            <link>https://ansumanbhujabal.medium.com/docker-named-volumes-what-when-how-for-begginers-18efc3dd3028?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/18efc3dd3028</guid>
            <category><![CDATA[docker-volume]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Tue, 07 May 2024 15:51:42 GMT</pubDate>
            <atom:updated>2024-05-07T15:51:42.451Z</atom:updated>
            <content:encoded><![CDATA[<h3>Docker Named Volumes What , When &amp; How for Begginers</h3><p>Docker Volume with Code snippets and real life use case</p><p>Docker container are <strong><em>non-persistent</em></strong> by default in nature</p><blockquote><strong><em>non-persistent — If data is stored inside container and the container is removed or deleted then all the data and configuration will be missed too.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SV2qwzIg_sw2nA5E" /><figcaption>Photo by <a href="https://unsplash.com/@rubaitulazad?utm_source=medium&amp;utm_medium=referral">Rubaitul Azad</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Docker Volumes</h3><ul><li>Volumes provide a way to persist data generated by and used by Docker containers.</li><li>One of the benefits of Docker Volumes is that they can be used to <strong>store data separately from the container itself</strong>. This means that even if a container is deleted, the data stored in the volume will remain.</li></ul><h3>Docker Volume Types</h3><p>In Docker, there are mainly <strong>two types of volumes</strong> that can be used to manage data persistency between containers and the host machine:</p><ol><li><strong>Named Volumes</strong>: Named volumes are explicitly created and given a user-defined name. They are managed by Docker, but users can specify the name when creating the volume. Named volumes offer easy data management and can be shared across multiple containers. They are the recommended approach for most use cases.</li><li><strong>Host-Bound Volumes</strong>: Host-bound volumes allow you to directly mount a directory from the host machine into the container. This provides easy access to host files from within the container, but it may create dependencies on the host environment.</li></ol><p>Now let’s start exploring Named Volumes in details .</p><h3>Named Volumes</h3><p>Named volumes are explicitly created and given a user-defined name. They are managed by Docker, but the user can specify the name when creating the volume. Named volumes offer easy data management and can be shared across multiple containers. They are the recommended approach for most use cases.</p><p><strong>Creating a named volume:</strong></p><pre>docker volume create loginvolume</pre><p>loginvolume: volume name</p><pre>docker volume ls<br>DRIVER    VOLUME NAME<br>local     loginvolume</pre><pre>docker volume inspect loginvolume<br>[<br>    {<br>        &quot;CreatedAt&quot;: &quot;2024-03-30T11:08:03Z&quot;,<br>        &quot;Driver&quot;: &quot;local&quot;,<br>        &quot;Labels&quot;: null,<br>        &quot;Mountpoint&quot;: &quot;/var/lib/docker/volumes/loginvolume/_data&quot;,<br>        &quot;Name&quot;: &quot;loginvolume&quot;,<br>        &quot;Options&quot;: null,<br>        &quot;Scope&quot;: &quot;local&quot;<br>    }<br>]</pre><p><strong>Remember</strong> — <strong><em>“/var/lib/docker/volumes/loginvolume/_data”</em></strong></p><p>is called as mount point and can be only accessed from inside a docker container</p><p>Below shell checks if we can access or not.</p><pre>sudo cd /var/lib/docker/volumes/loginvolume/_data<br>[sudo] password for ansuman:<br>Sorry, try again.</pre><pre>docker run -d -p 80:80 --name loginapp -v loginvolume:/usr/share/nginx/html nginx<br><br>-v - stands for volume <br>loginvolume - volume name <br>/usr/share/nginx/html nginx - Data path of nginx , <br>it varies from image to image</pre><ul><li>-v — stands for volume</li><li>loginvolume — volume name</li><li>/usr/share/nginx/html nginx — Data path of nginx</li><li>Data path varies from image to image</li></ul><p>Going inside docker volume via container as named volume can only be accessed from container</p><pre>docker exec -it loginapp bin/bash</pre><p>Let’s add some data inside our docker volume to test</p><pre>apt update<br>apt install git<br>git clone https://github.com/Ansumanbhujabal/login-2413.git<br>ls<br>cp login-2413/* /usr/share/nginx/html<br>cd /usr/share/nginx/html<br>ls<br>50x.html  README.md  img_avatar2.png  index.html<br>exit</pre><p>Output on port 80 should look like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*ldf7MzeJ4jZxIhB7E0mxyA.png" /><figcaption>Image of successful volume usage</figcaption></figure><p>Deleting volume after stopping and deleting the container</p><pre>docker volume rm loginvolume</pre><h3><strong>When</strong></h3><p>Named volumes in Docker are useful in various scenarios, including:</p><ol><li><strong>Sharing Data Between Containers</strong>: Named volumes can be easily shared among multiple containers. This is advantageous in microservices architectures where different containers need access to the same data.</li></ol><p><strong><em>example — </em></strong>you might have a database container and multiple application containers that all need access to the same database files stored in a named volume.</p><ol><li><strong>Development and Testing Environments</strong>: Named volumes are valuable in development and testing environments where you need to preserve data across container restarts or updates. Developers can use named volumes to mount code directories or database files into containers, allowing them to make changes to the code or data without affecting the underlying volume.</li><li><strong>Backup and Restore</strong>: Named volumes simplify the backup and restore process for Docker containers. Since data is stored separately from the container itself, you can easily create backups of named volumes and restore them as needed. This is crucial for ensuring data integrity and disaster recovery in production environments.</li></ol><h3><strong>Problem :</strong></h3><p>Volumes can’t be accessed outside containers , so becomes tidious for modification and update on data .</p><p>Less control as the configuration is managed by docker .</p><h3><strong>Solution</strong> :<strong>Host-Bound Volumes</strong></h3><p>we will explore in next blog .</p><p><strong>Connect with Me on GitHub and LinkedIn</strong></p><p>If you found this journey through Docker as fascinating as I did, let’s connect! You can explore the code and examples discussed in this blog on my GitHub repository. Also, if you’re interested in more devops-related insights and cloud adventures, follow me on LinkedIn.</p><p><a href="https://github.com/Ansumanbhujabal">GitHub </a>| <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/">LinkedIn</a></p><p>Let’s dive deeper into the data-driven world together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18efc3dd3028" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ansible setup on ubuntu local& remote (ec2)]]></title>
            <link>https://ansumanbhujabal.medium.com/ansible-setup-on-ubuntu-local-remote-ec2-c8dbda9583af?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8dbda9583af</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Fri, 15 Mar 2024 14:19:44 GMT</pubDate>
            <atom:updated>2024-03-15T14:19:44.949Z</atom:updated>
            <content:encoded><![CDATA[<p>How to use Ansible on ubuntu 20.04 from begginer pov</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QKr3JoGrn-cDaAVM" /><figcaption>Photo by <a href="https://unsplash.com/@6heinz3r?utm_source=medium&amp;utm_medium=referral">Gabriel Heinzer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7oOPfqBrAliP00s0Sftbvw.png" /></figure><h3>What Is Ansible</h3><p>Ansible is an open-source automation tool widely used for IT tasks such as</p><p><strong>. configuration management</strong></p><p><em>e.g-</em> suppose I want to configure a new service on many systems at a time</p><p><strong>. application deployment</strong></p><p><em>e.g- </em>suppose I want to run httpd Apache server and then run my simple application on top of it</p><p><strong>. task automation</strong></p><p><em>e.g-</em> suppose I want to clear up my log files of application in servers</p><p>It is designed to be</p><p><strong>. simple</strong></p><p><strong>. agentless</strong></p><p>uses SSH for connection</p><p><strong>. powerful</strong></p><p>enabling users to automate repetitive tasks efficiently. Ansible operates by connecting to nodes (servers or devices) over SSH or WinRM (Windows Remote Management) and executing tasks using YAML-based playbooks.</p><h3>A Little about architecture</h3><ul><li><strong>Controller Node</strong>: The system where Ansible is installed and from where automation tasks are orchestrated. The controller node manages the inventory of target hosts and executes playbooks to configure and manage them.</li></ul><p><strong>&gt;&gt;&gt;</strong> Here our Ubuntu 20.04 System</p><p><strong>&gt;&gt;&gt;</strong> control machine can be only a linux machine</p><ul><li><strong>Inventory</strong>: The inventory file contains a list of managed nodes that Ansible will work with. It can be static or dynamic, and it can organize hosts into groups to simplify management and orchestration.</li><li><strong>Playbooks</strong>: Playbooks are written in YAML and define a set of tasks to be executed on managed nodes. They specify the state the system should be in, rather than the exact steps to get there, making them idempotent and easy to understand.</li></ul><p><strong>Installation</strong></p><pre>$ sudo apt update</pre><pre>$ sudo apt install software-properties-common</pre><pre>$ sudo add-apt-repository - yes - update ppa:ansible/ansible</pre><pre>$ sudo apt install ansible</pre><p>Add the following lines</p><p>Have to add utf encoding in <strong>/sudo nano /etc/default/locale</strong></p><pre>LANG=&quot;en_US.UTF-8&quot;<br>LC_CTYPE=&quot;en.US.UTF-8&quot;</pre><p>Then</p><pre>sudo update-locale LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8</pre><p><strong>Ansible Version</strong></p><pre>ansible - version</pre><p><strong>for checking installation</strong></p><pre>ansible localhost -m ping</pre><p><strong>The output should look like</strong></p><pre>localhost | SUCCESS =&gt; {<br>&quot;changed&quot;: false,<br>&quot;ping&quot;: &quot;pong&quot;<br>}</pre><p><strong>Then</strong></p><pre>cd /etc/ansible/<br>ls -lrt # for checking the config<br>less ansible.config<br>ansible-config init - disabled &gt; ansible.cfg</pre><p>from ansible config , generate one sample config</p><h3>Running on local</h3><ul><li>Create a Playbook with host as localhost</li><li>It may ask sudo previliges , so user will <strong>become: true</strong></li></ul><pre>---<br>- name: Ansu&#39;s first Playbook<br>  hosts: localhost<br><br><br>  tasks:<br>  - name: Test Connectivity<br>    ping:<br>  - name: &quot;Print Output&quot;<br>    debug:  msg=&quot;Wroking Fine&quot;</pre><p><strong>Synatx check for a Playbook YAML file</strong></p><pre>ansible-playbook - syntax-check first_playbook.yaml</pre><p><strong>Running</strong></p><pre>ansible-playbook first_playbook.yaml</pre><p>The Output should look like this</p><pre>ansuman@ansuman:/opt/CodeRepo/DevopsBootCamp/Ansible/playbooks$ ansible-playbook first_playbook.yaml <br>[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match &#39;all&#39;</pre><pre>PLAY [Ansu&#39;s first Playbook] *****************************************************************************************************************************************</pre><pre>TASK [Gathering Facts] ***********************************************************************************************************************************************<br>ok: [localhost]</pre><pre>TASK [Test Connectivity] *********************************************************************************************************************************************<br>ok: [localhost]</pre><pre>TASK [Print Output] **************************************************************************************************************************************************<br>ok: [localhost] =&gt; {<br>    &quot;msg&quot;: &quot;Wroking Fine&quot;<br>}</pre><pre>PLAY RECAP ***********************************************************************************************************************************************************<br>localhost                  : ok=3    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0</pre><h3>Running for Remote Server</h3><p>. Create an ec2 instance in aws via console or cli with key token</p><p>. SSH should be allowed there from anywhere</p><p>. Download the pem or ppk file</p><p>. get the public ip</p><pre>cd /etc/ansible</pre><pre>sudo chmod 777 hosts<br>less hosts</pre><p><strong>Add the target ip in the hosts file</strong></p><pre># This is the default ansible &#39;hosts&#39; file.<br>#<br># It should live in /etc/ansible/hosts<br>#<br>#   - Comments begin with the &#39;#&#39; character<br>#   - Blank lines are ignored<br>#   - Groups of hosts are delimited by [header] elements<br>#   - You can enter hostnames or ip addresses<br>#   - A hostname/ip can be a member of multiple groups<br><br># Ex 1: Ungrouped hosts, specify before any group headers:<br>43.207.91.249<br></pre><p>confirm via</p><pre>ansible-inventory --list</pre><p>The output should look like</p><pre>{<br>    &quot;_meta&quot;: {<br>        &quot;hostvars&quot;: {}<br>    },<br>    &quot;all&quot;: {<br>        &quot;children&quot;: [<br>            &quot;ungrouped&quot;<br>        ]<br>    },<br>    &quot;ungrouped&quot;: {<br>        &quot;hosts&quot;: [<br>            &quot;172.31.12.13&quot;<br>        ]<br>    }<br>}</pre><p><strong>connection check</strong></p><pre>ansible -i hosts all -m ping --private-key=/opt/CodeRepo/Documentation/user5.pem -u ec2-user<br>(pings all the host present in the host file )</pre><p><strong>Added the public ip in a playbook</strong></p><pre>---<br>- name: Ansu&#39;s first Playbook<br>  hosts:  43.207.91.249<br><br><br>  tasks:<br>  - name: Test Connectivity<br>    ping:<br>  - name: &quot;Print Output&quot;<br>    debug:  msg=&quot;Wroking Fine&quot;</pre><p><strong>Then use</strong></p><pre>ansible-playbook -i hosts /opt/CodeRepo/DevopsBootCamp/Ansible/playbooks/ec2playbook.yaml --private-key=/opt/CodeRepo/Documentation/user5.pem -u ec2-user</pre><p>in above command :</p><p>hosts</p><p><strong><em>path to playbook &gt;&gt;</em></strong></p><p>/opt/CodeRepo/DevopsBootCamp/Ansible/playbooks/ec2playbook.yaml</p><p><strong><em>path to pem file&gt;&gt;</em></strong></p><p>— private-key=/opt/CodeRepo/Documentation/user5.pem</p><p><strong><em>u ec2-user name&gt;&gt;</em></strong></p><p>-u ec2-user</p><p><strong>Connect with Me on GitHub and LinkedIn</strong></p><p>If you found this journey through Ansible as fascinating as I did, let’s connect! You can explore the code and examples discussed in this blog on my GitHub repository. Also, if you’re interested in more devops-related insights and cloud adventures, follow me on LinkedIn.</p><p><a href="https://github.com/Ansumanbhujabal">GitHub </a>| <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/">LinkedIn</a></p><p>Let’s dive deeper into the data-driven world together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8dbda9583af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Label Encoding in Machine Learning]]></title>
            <link>https://ansumanbhujabal.medium.com/label-encoding-in-machine-learning-267eb4c928f1?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/267eb4c928f1</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Wed, 24 Jan 2024 05:33:32 GMT</pubDate>
            <atom:updated>2024-01-24T05:33:32.526Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Introduction</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9_Hg5rSHl9kkobbe" /><figcaption>Photo by <a href="https://unsplash.com/@marius?utm_source=medium&amp;utm_medium=referral">Marius Masalar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Machine Learning, with its vast landscape of concepts, tools, and techniques, can be overwhelming for beginners. In this comprehensive guide, we’ll shed light on a fundamental preprocessing technique: Label Encoding. We’ll explore the intricacies of Label Encoding, its relevance in Machine Learning workflows, and provide hands-on examples for a thorough understanding.</p><h3>Understanding Label Encoding</h3><p><strong>What is Label Encoding?</strong></p><p>Label Encoding is a method employed to transform categorical data into a numerical format, facilitating the integration of non-numeric data into machine learning models. The primary goal is to convert labels into a numeric representation, enabling algorithms to process the information effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cloLUaQ-uy82AqKoS2mcmg.png" /><figcaption>Encoding image by Chris Albon</figcaption></figure><blockquote><strong><em>example</em></strong>- Suppose in a <strong>Diabetes</strong> dataset there are<strong> 2 features</strong>, one person can be <strong>diabetic or non-diabetic </strong>. So, after Label Encoding we can label the <strong>diabetic as 0 and non-diabetic as 1 .</strong></blockquote><p><strong>Why Label Encoding?</strong></p><p>Machine Learning algorithms often require numerical input for training and prediction. Label Encoding serves as a bridge, allowing models to interpret and analyze categorical data accurately. It simplifies the process of working with features that denote different categories or groups.</p><h3>The Label Encoding Process</h3><p><strong>Step 1: Identifying Categorical Features</strong></p><p>Begin by recognizing features in your dataset that fall into the categorical domain. These features represent distinct categories or groups, such as <em>‘Color,’ ‘Type,’ or ‘Genre.’</em></p><blockquote><strong><em>example</em></strong>- <strong>diabetic</strong> and <strong>non-diabetic</strong></blockquote><p><strong>Step 2: Importing Libraries</strong></p><p>In Python, two commonly used libraries for Label Encoding are scikit-learn’s<strong> LabelEncoder</strong> and pandas’ <strong>factorize </strong>function. Import these tools into your machine learning environment to leverage their functionality.</p><pre>from sklearn.preprocessing import LabelEncoder<br>label_encoder = LabelEncoder()</pre><p><strong>Step 3: Instantiating Label Encoder</strong></p><p>Create an instance of the Label Encoder to utilize its methods.</p><p><strong>Step 4: Fitting and Transforming</strong></p><p>Apply the <strong>fit_transform </strong>method to your categorical column, converting the labels into numerical equivalents.</p><pre># Assuming &#39;column_name&#39; is your categorical column<br>data[&#39;column_name_encoded&#39;] = label_encoder.fit_transform(data[&#39;column_name&#39;])</pre><p><strong>Example: Bringing Theory to Life</strong></p><p>Consider a <strong><em>Color</em></strong> column with categories <strong>[‘Red’, ‘Blue’, ‘Green’]</strong>. After Label Encoding,<strong><em> ‘Red’ </em></strong>might become<strong> 0</strong>,<strong><em> ‘Blue’</em></strong> becomes <strong>1 ,</strong>and <strong>‘Green’</strong> becomes <strong><em>2</em></strong>.</p><pre>from sklearn.preprocessing import LabelEncoder<br>#finding number of labels<br>dataset[&#39;columnname&#39;].value_counts()<br><br>#initializing encoder<br>label_encoder = LabelEncoder()<br><br>#encoding<br>label=label_encoder.fit_transform(dataset[&#39;columnname&#39;])<br><br>#joining<br>dataset[&#39;newcolumnname&#39;]=label<br><br>#verify<br>dataset[&#39;newcolumnname&#39;].value_counts()</pre><p>While Label Encoding is a powerful preprocessing tool, it comes with some caveats:</p><p><strong>Ordinality Assumption</strong></p><p>Label Encoding implies ordinality, suggesting a meaningful order in the numeric values assigned to categories. This assumption might not hold true for nominal categories without a specific order.</p><p><strong>Impact on Algorithms</strong></p><p>Some machine learning algorithms may misinterpret encoded values as having a numerical relationship, potentially affecting model performance. Always be mindful of the nature of your data and the requirements of your chosen algorithm.</p><p><strong>Conclusion: Mastering Label Encoding</strong></p><p>Label Encoding is a valuable technique for handling categorical data in machine learning. As you delve into this method, it’s crucial to comprehend its nuances and consider alternative techniques for different scenarios. A thoughtful approach to preprocessing will enhance the robustness of your machine learning models. Dive in, experiment, and elevate your understanding of Label Encoding for a more effective and nuanced machine learning journey. Happy coding! 🚀✨</p><p><strong>Connect with Me on GitHub and LinkedIn</strong></p><p>If you found this journey through Label Encoding as fascinating as I did, let’s connect! You can explore the code and examples discussed in this blog on my GitHub repository. Also, if you’re interested in more data-related insights and machine learning adventures, follow me on LinkedIn.</p><p><a href="https://github.com/Ansumanbhujabal">GitHub </a>| <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/">LinkedIn</a></p><p>Let’s dive deeper into the data-driven world together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=267eb4c928f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Git Groovy: Forks, Commits, and PRs]]></title>
            <link>https://ansumanbhujabal.medium.com/git-groovy-forks-commits-and-prs-7cfbc50037cb?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/7cfbc50037cb</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[codingbootcamp]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Wed, 11 Oct 2023 13:49:05 GMT</pubDate>
            <atom:updated>2023-10-11T13:49:05.465Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1QRdUV2P8VHl45H9" /><figcaption>Photo by <a href="https://unsplash.com/@synkevych?utm_source=medium&amp;utm_medium=referral">Roman Synkevych</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>“The Easy Peasy Guide for GitHub Newbies!”</h3><p><strong>Introduction</strong></p><p>GitHub, the world’s leading platform for version control and collaboration, can be a little overwhelming for beginners. If you’re new to GitHub and looking to make your first contribution, you’re in the right place! In this step-by-step guide, I’ll walk you through forking a repository, making changes, committing those changes, and creating a pull request — all using GitHub Desktop. I’ll make sure to include reference images to make the process as beginner-friendly as possible. Let’s get started!</p><p><strong>Step 1: Forking a Repository</strong></p><ol><li><strong>Go to the repo:</strong> First, navigate to the repository you want to contribute to on GitHub by clicking the repo link .</li><li><strong>Fork the Repository:</strong> In the top right corner of the repository page, click the “Fork” button. This creates a personal copy of the repository on your GitHub account.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nxliaUtDfr_xVY2JvOyZ0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/1*NimWqYcr8XuThKOfeM9SGA.png" /><figcaption>Click on fork button</figcaption></figure><p>Make sure u select your profile here as owner below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*jg02Ua0uZbqd4oNCr8ykGQ.png" /><figcaption>Click on create Fork , Make sure u are owner of it</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XISwFW5vxnDi-M-go2Q2g.png" /><figcaption>It is correct , if u see the forked from”” message below repo name</figcaption></figure><p>It is correct , if u see the forked from “ “message below repo name</p><p><strong>3. Clone the Repository:</strong> On your forked repository, click the green “Code” button and then select “Open with GitHub Desktop.” This will launch GitHub Desktop and clone the repository to your local machine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*d2QMYq0iH_47Y-usAPUQ8A.png" /><figcaption>click on code , then open with Git hub desktop</figcaption></figure><p>then a popup will come and you have to click on</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/1*JxItyX2qVEiqG4nSqcJpyQ.png" /><figcaption>open with Git hub desktop</figcaption></figure><p>once it opens , you will see something like this below.<strong> Choose the path correctly making sure there aren’t any other git files existing</strong>. I suggest click on choose path option then go to Your any Drive like C and then create there a new empty folder by right click and select that .</p><p>Then click on the clone button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bWZ5UTQpJWeBx15P9bewEg.png" /><figcaption>choose a path according to your system then clone</figcaption></figure><p>then choose the below option to contribute to the parent or original repo then click on continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*lBqBBBNeJ5boxOh3CIFoBg.png" /><figcaption>choosing parent repo</figcaption></figure><p>Then You can see something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o7MjcBtmpdfAKU-zEcOv-w.png" /><figcaption>success cloned</figcaption></figure><p><strong>Step 2: Making Changes</strong></p><ol><li><strong>Make Your Changes:</strong> Using your favorite code editor, make the necessary changes to the files in the repository.</li></ol><p>Suppose for this , we will take one simple example of c<strong>reating a text file and pushing it .</strong></p><p>first , click on the option of open with windows <strong>explorer.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*HhyMB9IEA9mj0yv8emRWRw.png" /><figcaption>example change</figcaption></figure><p>Then it will take You to your files like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*32Mnoko-yu99AR0xzUwMpA.png" /><figcaption>folder structure</figcaption></figure><blockquote>Then <strong>Right-click there and add a new text file</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zv_ddBEgyKGyUykRsJRXuA.png" /><figcaption>new file</figcaption></figure><p>It would look like something this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*VN03f3hJTfSo_Nn1_DRhnA.png" /><figcaption>file change</figcaption></figure><p><strong>Step 3:Open Your Git hub Desktop now</strong></p><p>then , you should see something like this . the changes you made should show here also.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*8UxKuqXNuYe34U0afuPAUQ.png" /><figcaption>changes git hub</figcaption></figure><p><strong>Step 4: Committing Your Changes</strong></p><ol><li><strong>Review Changes:</strong> In GitHub Desktop, you’ll see the changes you’ve made. Review them to ensure everything looks correct.</li><li><strong>Write a Commit Message:</strong> Write a descriptive commit message explaining the changes you made. It helps others understand your contributions.</li></ol><blockquote><strong>Note- You can add and modify the commit message present in bottom above the description , Commit message is compulsory , it can not be blank . Description is optional.</strong></blockquote><p>Once you clicked on commit to main , you should see something like this , and then click on push origin</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6a3ksXujkfBxAu0S8qoruw.png" /><figcaption>push origin</figcaption></figure><p><strong>Step 5: Creating a Pull Request</strong></p><p>First go to your Git hub account and then to the forked repo of yours. then</p><ol><li><strong>Sync Your Fork:</strong> Before creating a pull request, click the “Sync” button in GitHub Desktop to ensure your fork is up-to-date with the original repository.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oMaaFd0ppZoKHOkfNUOCbQ.png" /><figcaption>sync fork</figcaption></figure><ol><li><strong>Create a Pull Request:</strong> In your forked repository on GitHub, click the <strong>“Contribute”</strong> button then on <strong>Open a Pull request </strong>button .</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*laLxS9FoE9lylc8VQERJqg.png" /></figure><ol><li><strong>Create Pull Request:</strong> Click the <strong>“Create pull request”</strong> button. Write a detailed description of your changes and why they’re beneficial. Then, click “Create pull request” to submit it.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1oYIgcvE3A9sS-uqJ--xEg.png" /><figcaption>pull request</figcaption></figure><p>Okay , cool . Now you should see something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nhUEMNoV4GJ0ah043nonaQ.png" /><figcaption>done PR</figcaption></figure><p><strong>Conclusion</strong></p><p>Well, folks, you did it! You’ve navigated the GitHub maze like a coding ninja. Now, go ahead and click that ‘Follow’ button here on the blog if you found our witty banter and tech wisdom delightful.</p><p>And guess what? If you follow me on GitHub, you might just learn to fork repositories with your eyes closed (don’t worry, we’re kidding… or are we?). Go ahead, hit that ‘Follow’ button on GitHub too, because why not? You never know when you might need a virtual high-five or a funny tech meme in your feed.</p><p>But wait, there’s more! For those of you looking to expand your professional network faster than your code compiles, connect with me on LinkedIn. I promise to fill your feed with more wisdom, humor, and the occasional cat picture to brighten your day.</p><p>So, what are you waiting for? Follow, like, and connect, and let’s embark on this coding adventure together. Happy coding, and may your bugs be as easy to fix as a typo in your favorite meme! 🚀😄</p><p><a href="https://github.com/Ansumanbhujabal">GitHub </a>| <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/">LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7cfbc50037cb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Standardization in ML]]></title>
            <link>https://ansumanbhujabal.medium.com/data-standardization-in-ml-2418fd7708de?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/2418fd7708de</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[standardization]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Wed, 27 Sep 2023 17:57:40 GMT</pubDate>
            <atom:updated>2023-09-27T17:57:40.833Z</atom:updated>
            <content:encoded><![CDATA[<p>Basic process for data standardization for ML</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*a_UCvXSZE-RGy4-c" /><figcaption>Photo by <a href="https://unsplash.com/@claybanks?utm_source=medium&amp;utm_medium=referral">Clay Banks</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Data standardization is a crucial preprocessing step in machine learning. It involves transforming our dataset to have a consistent scale and distribution, which can improve the performance of many machine learning algorithms.</p><p>In simple term it means processing our data into a common format before feeding it to machine learning algorithm.</p><figure><img alt="Data Standardization in brief by Ansuman" src="https://cdn-images-1.medium.com/max/473/1*j7QX_8Lw0amYQaR_IZIzJg.png" /><figcaption>Data Standardization in brief by Ansuman</figcaption></figure><p>The two common techniques for data standardization are <strong>Min-Max scaling</strong> and <strong>z-score normalization.</strong></p><p><strong>Min-Max Scaling</strong>: Min-Max scaling transforms our data into a specific range, usually [0, 1]. It linearly scales the data, so the minimum value becomes 0, and the maximum value becomes 1, while all other values are scaled proportionally in between.</p><p><strong>Z-Score Normalization (Standardization)</strong>: Z-score normalization transforms our data so that it has a mean of 0 and a standard deviation of 1. This method is particularly useful when your data follows a normal distribution.</p><p>Here are the steps in brief for data standardization.</p><ol><li>Finding Data Standard Deviation:</li></ol><pre>print(dataset.data.std()) // If o/p is 1 , then no devation</pre><p>2. Standard Scaler:</p><pre>from sklearn.preprocessing import StandardScaler// Import<br>scaler= StandardScaler()// Initialization</pre><p>3.Data fitting and Standardization:</p><p>Fit the scaler to your training data, and then transform both the training and test data using the fitted scaler. This step calculates the mean and standard deviation from the training data and applies the same transformation to both sets.</p><pre>scaler.fit(x_train)</pre><p>4. Data Transformation:</p><pre>x_train_standardized= scaler.transform(x_train)</pre><p>5.Data Standardizing of test based on train :</p><p>Use the same scaler that you fitted on the training data to standardize the test data. This ensures that the scaling parameters (mean and standard deviation) are consistent between the training and test sets.</p><pre>x_test_standardized= scaler.transform(x_test)</pre><p>6.Checking Standard Deviation</p><pre>print(x_train_standardized.std())<br>print(x_test_standardized.std())</pre><p>By following these steps, you ensure that your data is standardized (mean-centered and scaled) for use in machine learning models, making it easier for algorithms to converge and perform effectively, especially when they are sensitive to feature scales.</p><pre>from sklearn.preprocessing import StandardScaler<br><br># Assuming you have your dataset loaded as x_train and x_test<br><br># Step 1: Initialization<br>scaler = StandardScaler()<br><br># Step 2: Data Fitting and Standardization for the training set<br>scaler.fit(x_train)<br>x_train_standardized = scaler.transform(x_train)<br><br># Step 3: Data Standardization of the test set based on the training set<br>x_test_standardized = scaler.transform(x_test)<br><br># Step 4: Checking Standard Deviation<br>print(x_train_standardized.std())<br>print(x_test_standardized.std())</pre><p><strong>Conclusion: Data Gets a Makeover</strong></p><p>In the dazzling world of machine learning, even data deserves a makeover. Data standardization is like dressing up your dataset for a fancy soirée. It’s all about giving your numbers a fresh coat of paint and ensuring they waltz into the world of algorithms looking their best.</p><p>Picture this: your data, once a wild jungle of numbers, now marches in wearing tuxedos and ball gowns, thanks to Min-Max scaling and Z-Score normalization. The rowdy outliers have been tamed, and everyone’s dancing to the same beat, be it [0, 1] or the rhythmic heart of a mean 0 and standard deviation 1.</p><p>But don’t just take my word for it; the data itself can vouch for its transformation. It boasts about its newfound uniformity, flashing a standard deviation of 1 like a badge of honor.</p><p>So, remember, before you let your algorithms loose on your dataset, treat it to a spa day of standardization. Your models will thank you for the harmonious dance they’ll perform.</p><p><strong>Stay Standardized, Data Enthusiasts!</strong></p><p><strong>Connect with Me on GitHub and LinkedIn</strong></p><p>If you found this journey through data standardization as fascinating as I did, let’s connect! You can explore the code and examples discussed in this blog on my GitHub repository. Also, if you’re interested in more data-related insights and machine learning adventures, follow me on LinkedIn.</p><p><a href="https://github.com/Ansumanbhujabal">GitHub </a>| <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala/">LinkedIn</a></p><p>Let’s dive deeper into the data-driven world together!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2418fd7708de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning Basic-Handling Missing Values in Dataset]]></title>
            <link>https://ansumanbhujabal.medium.com/machine-learning-basic-handling-missing-values-in-dataset-ca81914380ed?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca81914380ed</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[handling-missing-values]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Mon, 11 Sep 2023 18:29:12 GMT</pubDate>
            <atom:updated>2023-09-11T18:29:12.368Z</atom:updated>
            <content:encoded><![CDATA[<p>How to handle missing values in datasets from the pov of ML</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*98x_4f-VdGYrsqCl5JFyeQ.png" /><figcaption>Panda Thumbnail By Ansuman</figcaption></figure><p>Datasets are essential element for training and testing any Machine Learning Model and before we feed the data to the model the data has to go through the pre-processing so, that our model can take the maximum accuracy and be more predictive .</p><p><strong>What is Missing Values:</strong></p><p>Generally we import datasets from any data collecting organization or any data managing site or database.</p><p>And we have to make sure that the dataset is complete and does not contain any <strong><em>Missing Value.</em></strong></p><blockquote><strong>Missing Value</strong>- When inside a datasets some <strong><em>features</em></strong> or<em> </em><strong><em>labels</em></strong> does not have any values they can create mis-calculation in our ML model.</blockquote><blockquote><strong>example</strong>- Suppose there is a dataset which contains data about Rocks and Mines. It has some features and the end labels says whether the object is Rock or Mine . Suppose in that dataset 50 Rock and 50 Mine data are there.</blockquote><blockquote>If there are some empty or missing value features field in Rock data and the labels are Rocks as usual. So when we will train our model based on this , the model will take empty as a special value and will lead to miss-prediction of model ultimately failing the model.</blockquote><p><strong>How to handle Missing Values:</strong></p><p><strong><em>step-1:</em></strong> Find Missing Values</p><pre>#dataset = your dataset in pandas framework<br>dataset.isnull.sum()<br></pre><p>this line of code will find each and every missing value field n the dataset.</p><p><strong><em>step-2:</em></strong> Handle</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*TmeYFZDHl0g1UTTtNXpT2A.png" /><figcaption>Handling Missing Values</figcaption></figure><p>There are 2 major ways we can handle missing values once we are sure.</p><ol><li><strong>Removing (Dropping)</strong>:</li></ol><p>Just simply removing those particular rows whose features are null or value is missing .</p><p>Better for larger datasets as it does not affect the dataset overall.</p><p>Not recommended for smaller datasets.</p><p><strong>code#</strong></p><pre>#drops all rows that has any feature missing values<br>dataset=dataset.dropna(how=&#39;any&#39;)<br></pre><p>2. <strong>Imputation:</strong></p><p>Replacing the missing values with some <strong>Statistical values<em>(mathematically and logically relevant)</em></strong> that won’t disturb and change the pattern of the dataset.</p><p>Generally we consider 3 statistical values in imputation method.</p><p><strong><em>Mean</em></strong></p><p><strong><em>Median</em></strong></p><p><strong><em>Mode</em></strong></p><p>Now let’s recall all these concepts from high school with the help of this image</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*Xcnx8LnraKyXS99kAX-a0g.png" /><figcaption>Mean By Ansuman</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Lnci2GGUYTu0GRGwhtoUw.png" /><figcaption>Median By Ansuman</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VE6CzefLE3USNBaOkrdjFA.png" /><figcaption>Mode By Ansuman</figcaption></figure><p>To know exactly what value to replace with, first we need to have a look at distribution pattern of missing value.</p><p><strong>code#</strong></p><pre>sns.displot(dataset.feature_name)<br></pre><ol><li><strong>Mean</strong>:</li></ol><ul><li><strong>When to Use</strong>: The mean is used to impute missing values when dealing with continuous or numeric data, such as age, income, or temperature.</li><li><strong>How it Works</strong>: Calculate the mean (average) of the available data points in the feature and use this value to fill in the missing entries.</li><li><strong>Example</strong>: If you have a dataset of ages and some entries are missing, you can calculate the mean age of the individuals with known ages and use that value to impute the missing ages.</li></ul><ol><li><strong>Mode</strong>:</li></ol><ul><li><strong>When to Use</strong>: The mode is used for imputation when dealing with categorical data or data with a limited set of distinct categories, like car types, colors, or city names.</li><li><strong>How it Works</strong>: Find the mode, which is the most frequently occurring value in the available data, and use it to replace the missing values.</li><li><strong>Example</strong>: If you have a dataset of car colors and some entries are missing, you can find the most common color among the known entries and use that color to impute the missing values.</li></ul><ol><li><strong>Median</strong>:</li></ol><ul><li><strong>When to Use</strong>: The median is a good choice for imputation when dealing with skewed data or outliers in continuous or numeric data. It’s less sensitive to extreme values than the mean.</li><li><strong>How it Works</strong>: Arrange the available data in ascending order and find the middle value. Use this middle value as the imputed value for missing entries.</li><li><strong>Example</strong>: If you have a dataset of incomes, and there are some extreme outliers (very high or low incomes), the median can provide a more robust imputation method than the mean.</li></ul><p>In the sns plot if the outliers(Different from general value pattern) are more significant, it is known as skewed.</p><p>And the mean does not become significant enough to show relation as it is highly manipulated by outliers.</p><p>So median and modes are choices we left with.</p><p><strong>code#</strong></p><pre>dataset[&#39;feature_name&#39;].fillna(dataset[&#39;feature_name&#39;].median(), inplace=True)</pre><h3><strong>Conclusion:</strong></h3><p>🤖🤣 In conclusion, handling missing values in your datasets is like trying to find your car keys in a pile of laundry — you know they’re in there somewhere, but it can be a real challenge! But fear not, brave data explorer, armed with the power of mean, mode, and median, you can turn those missing values into found treasures for your machine learning models.</p><p>If you enjoyed this data-driven comedy show, don’t forget to give it a star on my GitHub repository and connect with me on LinkedIn to stay updated with more hilarious data adventures. Remember, in the world of machine learning, a good sense of humor can be just as important as a well-imputed dataset! 😄</p><p>Follow me on GitHub: <a href="https://github.com/Ansumanbhujabal/">GitHub</a>:</p><p>Connect with me on LinkedIn: <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala-30851922b/">LinkedIn</a>:</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca81914380ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI vs ML vs DL Basics You Should Know Before Starting..]]></title>
            <link>https://ansumanbhujabal.medium.com/ai-vs-ml-vs-dl-basics-you-should-know-before-starting-ef82392337c9?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef82392337c9</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Sat, 02 Sep 2023 15:22:33 GMT</pubDate>
            <atom:updated>2023-09-02T15:22:33.585Z</atom:updated>
            <content:encoded><![CDATA[<p>The basics and differences to know before learning ML</p><p>We will try to understand each of these basics in a simple and easy to grasp way.</p><p><strong>1.AI</strong></p><p>Inter Disciplinary Branch of Computer Science which deals with building <em>smart and intelligent machines.</em></p><blockquote>Inter Disciplinary — <em>Not limited to it’s development and usage in a particular field or branch.</em></blockquote><blockquote>smart and intelligent machines <em>— Machines that are capable of taking logical decisions on their own based on the input we provide to them.</em></blockquote><blockquote>example- Automated cars , self controlled jets</blockquote><p><strong>2.ML</strong>(Machine Learning)</p><p><em>Technique to implement AI</em> that machine can learn from data by themselves <em>without explicitly programmed</em> for each and every case.</p><p><strong>3.DL</strong>(Deep Learning)</p><p>Subfield of ML that uses <a href="https://www.javatpoint.com/artificial-neural-network"><em>Artificial Neural Network(ANN) </em></a>to learn from data.</p><figure><img alt="Veinn Of Ai" src="https://cdn-images-1.medium.com/max/653/1*k1H5DtihsWwbGKsEgENUDA.png" /></figure><h3>Types of ML</h3><p>Machine Learning can be generally divided into three major types .They are Supervised , Unsupervised and Reinforced</p><figure><img alt="Types Of ML Models" src="https://cdn-images-1.medium.com/max/783/1*HeW-NFV7veADIzY1kfzsBA.jpeg" /></figure><p>Their Differences lies in the procedures through which they are trained and processed.</p><p>To make it interesting adding an easy diagram made by me.</p><figure><img alt="ML Types and Differences" src="https://cdn-images-1.medium.com/max/776/1*ykB-UPeROw9G8sfAZSeKxQ.png" /></figure><h3><strong>Types Of Supervised Learning</strong></h3><p>For better understanding we can understand by this flowchart</p><figure><img alt="Types Of Supervised Learning" src="https://cdn-images-1.medium.com/max/802/1*UvAvwOLAq4Wu6sVYVR9Wxg.png" /><figcaption>Types Of Supervised Learning</figcaption></figure><h3>Types Of Un-Supervised Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*aNDeu4xXPw9tPAUJjNPDXQ.png" /><figcaption>Types Of Un-Supervised Learning</figcaption></figure><p><strong>Examples-</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*-N2nMM_XqAqi3L_9i-yUmw.png" /><figcaption>Examples Of Un-Supervised Learning</figcaption></figure><h3><strong><em>Conclusion</em></strong></h3><p>In conclusion, we’ve ventured through the labyrinth of AI, ML, and DL, deciphering the alphabet soup of modern technology. We’ve demystified AI as the interdisciplinary wizardry behind our smart machines. ML, our trusty sidekick, is the technique that teaches our machines to learn from data, sparing us from micromanaging their every move. And last but not least, DL, the caped crusader of ML, wields the power of Artificial Neural Networks to take our machines to unprecedented heights.</p><p><strong><em>Byee</em></strong></p><p>But hey, this isn’t the end of our journey. In the vast landscape of technology, there’s always more to explore, more acronyms to unravel, and more knowledge to gain. So, if you enjoyed this whirlwind tour of AI, ML, and DL, why not continue the adventure? Connect with me on <a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala-30851922b/"><em>LinkedIn</em></a> for updates, insights, and maybe the occasional meme to brighten your day. And if you’re feeling really ambitious, let’s collaborate on <a href="https://github.com/Ansumanbhujabal/"><em>GitHub</em></a> and build the next generation of intelligent machines together. After all, who knows what other exciting acronyms await our exploration in the future? 😉🤖</p><p><a href="https://www.linkedin.com/in/ansuman-simanta-sekhar-bhujabala-30851922b/">LinkedIn</a>:</p><p><a href="https://github.com/Ansumanbhujabal/">GitHub</a>:</p><p>Byeeee.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef82392337c9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mongo server Connection for beginner]]></title>
            <link>https://ansumanbhujabal.medium.com/mongo-server-connection-for-beginner-eebaa9884470?source=rss-3c61f3cddec------2</link>
            <guid isPermaLink="false">https://medium.com/p/eebaa9884470</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[mern-stack]]></category>
            <category><![CDATA[mongoose]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[Ansuman Bhujabala]]></dc:creator>
            <pubDate>Thu, 03 Aug 2023 07:57:00 GMT</pubDate>
            <atom:updated>2023-08-03T07:57:00.786Z</atom:updated>
            <content:encoded><![CDATA[<p>how to connect to MongoDB with express JS</p><p>First we need to import express and mongoose in our app.js file .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w9YBcQzAMqc0NJBV" /><figcaption>Photo by <a href="https://unsplash.com/@rubaitulazad?utm_source=medium&amp;utm_medium=referral">Rubaitul Azad</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>before that make sure to install them using npm or yarn .</p><pre>npm i express mongoose</pre><p>Before creating database and connecting it , we need to re-consider few things . MongoDB saves documents in Bison(Binary JSON) format .So, we can embed the documents inside one .</p><p>Mongoose uses schema to define the structure of the document (in simple , what are the fields we want to have).</p><p>Then we create one model based on that schema and use it to connect to the database .</p><p>For connection Mongo gives us connection URL , here we have many options like localhost , atlas url. But for initial stage we will be going with localhost.</p><p>for visualization of data, we can use compass which has awesome UI to help us seeing data there .(Install it directly from Mongo Website).</p><p>Now let’s write code in app.js</p><pre><br>const express = require(&quot;express&quot;);<br>const mongoose = require(&quot;mongoose&quot;);<br>const app = express();<br><br>//Schema<br>const userSchema = new mongoose.Schema({<br>  name: String,<br>  email: String,<br>});<br>//Base<br>const User = mongoose.model(&quot;User&quot;, userSchema);<br>mongoose<br>  .connect(<br>    &quot;mongodb://localhost:27017&quot;<br>  )<br>  .then(() =&gt; {<br>    console.log(&quot;Connected to Db&quot;);<br><br>    //createnew user<br>    const newUser = new User({<br>      name: &quot;waka&quot;,<br>      email: &quot;@gmail&quot;,<br>    });<br>    //save<br>    return newUser.save();<br>  })<br>  .then(() =&gt; {<br>    console.log(&quot;saved successfully&quot;);<br>  })<br>  .catch((error) =&gt; {<br>    console.error(&quot;Failed&quot;);<br>  });</pre><p>here let’s breakdown each line</p><pre>const userSchema = new mongoose.Schema({<br>  name: String,<br>  email: String,<br>});</pre><p>created userSchema which contains the fields name and email and their type also .</p><pre>const User = mongoose.model(&quot;User&quot;, userSchema);</pre><p>created first model that whole DB will look like.</p><pre>mongoose<br>  .connect(<br>    &quot;mongodb://localhost:27017&quot;<br>  )</pre><p>connected to MongoDB , here the URL is default URL of mongo , now-a-days sometimes this is not working , instead of that we can use</p><pre>mongoose<br>  .connect(<br>    &quot;mongodb://0.0.0.0:27017&quot;<br>  )</pre><p>This also works the same .</p><p>Mongoose no longer accepts callback functions , so we will be using then block.</p><pre>.then(() =&gt; {<br>    console.log(&quot;Connected to Db&quot;);<br><br>    //createnew user<br>    const newUser = new User({<br>      name: &quot;waka&quot;,<br>      email: &quot;@gmail&quot;,<br>    });<br>    //save<br>    return newUser.save();<br>  })</pre><p>after connecting , we will log message “Connected to Db”</p><p>then , will create new user with the data as given there and it will be saved in database.</p><pre>.then(() =&gt; {<br>    console.log(&quot;saved successfully&quot;);<br>  })<br>  .catch((error) =&gt; {<br>    console.error(&quot;Failed&quot;);<br>  });</pre><p>if everything goes well , we will log the message “saved successfully”.</p><p>and catch block catches the error if any happens and we log “Failed”.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eebaa9884470" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>