<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vasista Reddy on Medium]]></title>
        <description><![CDATA[Stories by Vasista Reddy on Medium]]></description>
        <link>https://medium.com/@vasista?source=rss-6bbf0e698487------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JLDogEMjKY9boDrfRaawVQ.jpeg</url>
            <title>Stories by Vasista Reddy on Medium</title>
            <link>https://medium.com/@vasista?source=rss-6bbf0e698487------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 04:42:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vasista/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How AI Remembers in an Agentic World]]></title>
            <link>https://vasista.medium.com/how-ai-remembers-in-an-agentic-world-3eb89353c165?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/3eb89353c165</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agentic-memory]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Thu, 08 Oct 2026 15:11:03 GMT</pubDate>
            <atom:updated>2026-10-08T15:12:15.042Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong><em>The Core Problem: Why LLMs Have “Amnesia”</em></strong></blockquote><p>By default, LLMs are <strong>stateless</strong>. Every API call starts with a blank slate. While context windows have grown dramatically (1M+ tokens), context is not memory:</p><ul><li><strong>Context is Working Memory (RAM):</strong> Temporary, expensive, and dilutes attention over time (the “needle in a haystack” problem).</li><li><strong>Memory is Long-Term Storage (Hard Drive):</strong> Persistent, indexed, and retrievable across sessions and task lifecycles.</li></ul><p>Without persistent memory, an agent cannot adapt, personalize, or execute multi-day, complex agentic workflows.</p><h4>The Four Tiers of Agent Memory</h4><p>To build autonomous systems, we divide memory into four key categories:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tTAaQkj77VBzonU9-C14Mw.png" /></figure><h4>How Memory Flows in an Agentic Loop</h4><p>A standard Agent Memory Architecture follows a 4-step loop:</p><ol><li><strong>Ingestion &amp; Reflection:</strong> The agent observes the conversation or execution environment, extracts salient insights, and discards filler noise.</li><li><strong>Storage &amp; Categorization:</strong> Facts go to a Knowledge Graph/KV store; events go to an Episodic Vector DB; skills update the tool catalog.</li><li><strong>Retrieval (RAG + Hybrid):</strong> When a new prompt arrives, the agent uses hybrid retrieval (Dense Vector + Knowledge Graph + Recency Score) to fetch relevant memories.</li><li><strong>Context Injection:</strong> Retracted memories are dynamically injected into the LLM system prompt before action generation.</li></ol><h4>Challenges &amp; Frontiers in Agent Memory</h4><ul><li><strong>The Memory Decay Problem:</strong> Not all memories are equal. Older, irrelevant, or superseded facts need to be updated or forgotten (e.g., “I live in Seattle” →→ “I moved to NYC”).</li><li><strong>Memory Compression &amp; Consolidation:</strong> Summarizing long chains of execution into high-level abstractions, similar to how human REM sleep consolidates daily events into long-term learning.</li><li><strong>Privacy &amp; Security:</strong> Ensuring long-term persistent memory complies with privacy regulations (GDPR’s “Right to be Forgotten” applied to AI vector stores).</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3eb89353c165" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Securing Your Conversations: Creating Local Chatbots with GPT4All and LangChain for Data Privacy…]]></title>
            <link>https://medium.com/scrapehero/securing-your-conversations-creating-local-chatbots-with-gpt4all-and-langchain-for-data-privacy-1dfd211b43e8?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/1dfd211b43e8</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Tue, 10 Oct 2023 07:34:22 GMT</pubDate>
            <atom:updated>2023-10-12T10:00:14.618Z</atom:updated>
            <content:encoded><![CDATA[<h3>Securing Your Conversations: Creating Local Chatbots with GPT4All and LangChain for Data Privacy and Security</h3><p>In today’s digital age, where data privacy and security concerns loom large, organizations are increasingly seeking innovative solutions to protect sensitive information while still harnessing the power of AI-driven interactions.</p><p>In this blog, we delve into a fascinating synergy of two powerful technologies — <a href="https://github.com/nomic-ai/gpt4all"><strong>GPT4All</strong></a> and <a href="https://python.langchain.com/docs/get_started/quickstart"><strong>LangChain</strong></a> — to create local chatbots that not only enhance user experiences but also fortify the walls guarding your data.</p><p>Join us on this journey as we uncover the myriad advantages of employing local chatbots as sentinels for your digital conversations, ensuring the utmost in data privacy and security.</p><p>Local chatbots, sometimes referred to as on-premises or offline chatbots, can offer several benefits with regard to privacy and security of data compared to cloud-based chatbots.</p><p>Here are some of the advantages:</p><h3>Data Sovereignty</h3><p>One of the most significant benefits of local chatbots is data sovereignty. With local chatbots, all user data remains within the organization’s own infrastructure or premises.</p><p>This can be crucial for businesses or entities that need to comply with strict data privacy regulations, such as GDPR in Europe, as it ensures that sensitive user data doesn’t leave the organization’s jurisdiction.</p><h3>Reduced Data Exposure</h3><p>When using a cloud-based chatbot, data is transmitted over the internet to remote servers for processing. This data transmission can be vulnerable to interception or hacking during transit.</p><p>In contrast, local chatbots process data on-site, reducing the risk of exposure during transmission.</p><h3>Enhanced Control</h3><p>Organizations have greater control over the security measures and protocols in place when data is processed locally. They can implement their own security policies, encryption standards, and access controls to protect user data.</p><h3>Customized Security Measures</h3><p>Organizations can tailor security measures to their specific needs and the sensitivity of the data they handle.</p><p>This customization can include firewalls, intrusion detection systems, and encryption methods that align with the organization’s security policies.</p><h3>Offline Operation</h3><p>Local chatbots can continue to operate even when there is no internet connection. This can be crucial in scenarios where internet connectivity is intermittent or unreliable, ensuring uninterrupted service.</p><h3>Isolation from Cloud Vulnerabilities</h3><p>Cloud-based chatbots may be susceptible to cloud-specific vulnerabilities and attacks. Local chatbots are generally less exposed to these types of risks since they don’t rely on external cloud infrastructure.</p><h3>Compliance with Industry Regulations</h3><p>Certain industries, such as healthcare and finance, have strict regulatory requirements for data security and privacy.</p><p>Local chatbots can help organizations in these sectors maintain compliance by keeping data within their controlled environments.</p><p>Let’s dive into the practical aspects of creating a chatbot using GPT4All and LangChain.</p><h3>Installation &amp; Setup</h3><pre>Create a virtual environment and activate it.<br><br>python -m venv &lt;venv&gt;<br>&lt;venv&gt;\Scripts\Activate.ps1</pre><p>There are many ways to set this up. We will test wit<strong>h </strong><a href="https://github.com/nomic-ai/gpt4all"><strong>GPT4All</strong></a><strong> </strong>and PyGPT4All libraries. <a href="https://nomic-ai.github.io/pygpt4all/"><strong>PyGPT4All</strong></a><strong> </strong>is the Python CPU inference for GPT4All language models.</p><pre>Installing gpt4all<br><br>pip install gpt4all</pre><p>Test it out! In a Python script or console:</p><pre>from gpt4all import GPT4All<br>model = GPT4All(&#39;D:\gpt4all_langchain_chatbots\models\gpt4all-converted.bin&#39;)<br>for token in model.generate(&quot;Once upon a time&quot;, n_predict=55):<br>    print(token, end=&#39;&#39;, flush=True)</pre><p>Similarly, pygpt4all can be installed using pip. However, this project has been archived and merged into gpt4all</p><pre>Pygpt4all Code:<br><br>from pygpt4all.models.gpt4all import GPT4All<br><br>def new_text_callback(text):<br>    print(text, end=&quot;&quot;)<br><br>model = GPT4All(&#39;D:\gpt4all_langchain_chatbots\models\gpt4all-converted.bin&#39;)<br>model.generate(&quot;Once upon a time, &quot;, n_predict=55, new_text_callback=new_text_callback)<br><br>Output:</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/0*JivQ8l7CtkNp48i8" /></figure><p>I have downloaded the model from <a href="https://huggingface.co/mrgaang/aira/blob/main/gpt4all-converted.bin"><strong>here</strong></a><strong> </strong>because of latency and size constraints. GPT4All has the best-performing state-of-the-art models to replace it. I will provide a comparison later in the post.</p><p>Models can be downloaded from <a href="https://gpt4all.io/index.html"><strong>here</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/0*VPohoqKmybn8R8eF" /></figure><pre># Installing langchain<br>      pip install langchain</pre><p>Let’s test a sample code of langchain to confirm the installation</p><pre>from langchain import PromptTemplate, LLMChain<br>from langchain.llms import GPT4All<br>from langchain.callbacks.manager import CallbackManager<br>from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler<br><br>local_path = &#39;D:\gpt4all_langchain_chatbots\models\gpt4all-converted.bin&#39; <br>callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])<br><br>template = &quot;&quot;&quot;Question: {question}<br><br>Answer: Let&#39;s think step by step.<br><br>&quot;&quot;&quot;<br><br>prompt = PromptTemplate(template=template, input_variables=[&quot;question&quot;])<br>llm = GPT4All(model=local_path, callback_manager=callback_manager, verbose=True)<br>llm_chain = LLMChain(prompt=prompt, llm=llm)<br><br>question = &quot;What NFL team won the Super Bowl in the year Justin Bieber was born?&quot;<br># question = input(&quot;Enter your question: &quot;)<br><br>llm_chain.run(question)<br><br>Output:</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fV6-g_jYTI8tCCj9" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YPNx_8rjnLGKllgU" /></figure><p>Up until this point, we have exclusively utilized the pre-existing gpt4all models. Moving forward, we will feed custom data to the model by embedding and indexing.</p><h3>Data Collection &amp; Preparation</h3><p>We have collected the <a href="https://www.scrapehero.com/store/product/mcdonalds-store-locations-in-canada/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science">McDonald’s restaurant location data</a> from <a href="https://www.scrapehero.com/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science">ScrapeHero</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NxQHuuuRQ-N9CqQI" /></figure><p>We need to convert this data into embeddings using the llama model. Using these embeddings we will create an index that will be used in a similarity match between the question and index documents.</p><p><strong>Note</strong>: <em>Visual Studio 2022, cpp, cmake installations are a must to prompt the question to langchain prompt template.</em></p><p>We have a <a href="https://github.com/imartinez/privateGPT"><strong>privateGPT</strong></a> package that effectively addresses our challenges. It is developed using LangChain, GPT4All, LlamaCpp, Chroma, and SentenceTransformers.</p><h3>Implementation &amp; Testing</h3><p>In the implementation part, we will be comparing two GPT4All-J models i.e., <a href="https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin"><strong><em>ggml-gpt4all-j-v1.3-groovy.bin</em></strong></a> and <a href="https://gpt4all.io/models/wizardlm-13b-v1.1-superhot-8k.ggmlv3.q4_0.bin"><strong><em>wizardlm-13b-v1.1-superhot-8k.ggmlv3.q4_0.bin</em></strong></a></p><p>Clone PrivateGPT repo and download the models into the ‘models’ directory. Modify the .env file to specify the model path. The McDonald’s restaurant data will be located in the ‘source_documents’ folder, while the custom CSV data will be converted into a VectorStore database.</p><p>Run the following command to ingest the custom data as indexes into the db.</p><pre>python ingest.py</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/0*i3-HHnQJfbco33-q" /></figure><p>It will create a db folder containing the local vectorstore.</p><p><strong>Note</strong>: <em>Throughout the ingest process, data remains within our local environment. We can perform ingestion without an internet connection, except for the initial setup, where we run the ingest script to download the embedding model.</em></p><p>The embedding model in our setup is <strong>all-MiniLM-L6-v2</strong>.</p><p>Test the model with questions, locally! Run this command</p><pre>python privateGPT.py</pre><p>Question 1: How recent is the available McDonald’s data scraped?</p><p>Output from ggml</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZnaCAr3gES5zGUZZ" /></figure><p>Output from wizardlm</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BvNnZY0JTyTYoero" /></figure><p>Question 2: how many stores available in the city “TORONTO”?</p><p>Output from ggml</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/764/0*uHBvL1K9ZJg441kp" /></figure><p>Output from wizardlm</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZFFOikInuWTFwwMR" /></figure><p>The Wizardlm model outperforms the ggml model. It is the latest and best-performing gpt4all model. The model performs well with more data and a better embedding model.</p><h3>Conclusion</h3><p>We now have experience in constructing local chatbots capable of running without internet connectivity to enhance data security and privacy using LangChain, GPT4All, and PrivateGPT. These models can achieve superior performance with larger datasets and improved embeddings.</p><p>However, it’s essential to note that local chatbots also come with their own set of challenges. These include higher initial setup costs, the need for dedicated infrastructure and maintenance, and potential limitations in terms of scalability compared to cloud-based solutions.</p><p>Organizations should carefully assess their specific requirements and consider the trade-offs between local and cloud-based chatbots when it comes to privacy and security.</p><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1dfd211b43e8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/securing-your-conversations-creating-local-chatbots-with-gpt4all-and-langchain-for-data-privacy-1dfd211b43e8">Securing Your Conversations: Creating Local Chatbots with GPT4All and LangChain for Data Privacy…</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Image Similarity Approaches in Python]]></title>
            <link>https://medium.com/scrapehero/exploring-image-similarity-approaches-in-python-b8ca0a3ed5a3?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8ca0a3ed5a3</guid>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[image-processing]]></category>
            <category><![CDATA[image-similarity]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Tue, 05 Sep 2023 06:22:24 GMT</pubDate>
            <atom:updated>2023-10-12T09:38:39.012Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wE29c8Su4anODTbqMdt4cQ.jpeg" /></figure><p>In a world inundated with images, the ability to measure and quantify the similarity between images has become a critical task. Whether it’s for image retrieval, content recommendation, or visual search, image similarity approaches play a pivotal role in modern applications.</p><p>Fortunately, Python offers a plethora of tools and libraries that make exploring and implementing these approaches accessible to developers and researchers alike. In this blog, we’ll delve into various image similarity techniques and demonstrate how to implement them using Python.</p><h3>Understanding Image Similarity</h3><p>Image similarity can be thought of as a numerical representation of how alike two images are in terms of their visual content. There are several dimensions along which images can be similar, such as color, shape, texture, and composition. Various mathematical and computational methods are employed to quantify these similarities, enabling us to compare and categorize images efficiently.</p><h3>Popular Image Similarity Approaches</h3><h4>Histogram-Based Approaches</h4><p>Histograms capture the distribution of pixel values in an image. By comparing the histograms of two images, you can measure their similarity.</p><p>The Histogram Intersection and Histogram Correlation metrics are commonly used for this purpose. Python’s <strong><em>OpenCV </em></strong>library provides tools to compute and compare histograms.</p><p>Here, we’ll demonstrate a simple example using Histogram-Based Approaches and the<strong><em> opencv-python</em></strong> library:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kqtBYL0MYD_IuKzuuBvPdQ.png" /></figure><p>We’ll be using these two images in the rest of the blog’s demonstration parts.</p><pre>import cv2<br># Load images<br>image1 = cv2.imread(image1)<br>image2 = cv2.imread(image2)<br>hist_img1 = cv2.calcHist([image1], [0, 1, 2], None, [256, 256, 256], [0, 256, 0, 256, 0, 256])<br>hist_img1[255, 255, 255] = 0 #ignore all white pixels<br>cv2.normalize(hist_img1, hist_img1, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)<br>hist_img2 = cv2.calcHist([image2], [0, 1, 2], None, [256, 256, 256], [0, 256, 0, 256, 0, 256])<br>hist_img2[255, 255, 255] = 0  #ignore all white pixels<br>cv2.normalize(hist_img2, hist_img2, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)<br># Find the metric value<br>metric_val = cv2.compareHist(hist_img1, hist_img2, cv2.HISTCMP_CORREL)<br>print(f&quot;Similarity Score: &quot;, round(metric_val, 2))<br># Similarity Score: 0.94</pre><h4>Structural Similarity Index (SSIM)</h4><p>SSIM is a widely used metric that assesses the structural similarity between two images. It considers luminance, contrast, and structure, giving a score between -1 (dissimilar) and 1 (identical). The scikit-image library in Python offers an SSIM implementation.</p><p>Here, we’ll demonstrate a simple example using SSIM and the <strong><em>scikit-image</em></strong> library:</p><pre>import cv2<br>from skimage import metrics<br># Load images<br>image1 = cv2.imread(image1)<br>image2 = cv2.imread(image2)<br>image2 = cv2.resize(image2, (image1.shape[1], image1.shape[0]), interpolation = cv2.INTER_AREA)<br>print(image1.shape, image2.shape)<br># Convert images to grayscale<br>image1_gray = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)<br>image2_gray = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)<br># Calculate SSIM<br>ssim_score = metrics.structural_similarity(image1_gray, image2_gray, full=True)<br>print(f&quot;SSIM Score: &quot;, round(ssim_score[0], 2))<br># SSIM Score: 0.38</pre><p>The main drawback of the SSIM approach compared to the Histogram approach is that the images have to be the same dimension. Even the similarity score is very low. We can do background subtraction and transparency removal from the images to improve the similarity score.</p><h4>Feature-Based Approaches</h4><p>These methods extract salient features from images, such as edges, corners, or key points. Techniques like Scale-Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF) identify distinctive points in images, which can then be compared across images.</p><p>The <strong><em>opencv-python</em></strong> library can be used for SIFT and SURF.</p><h4>Deep Learning-Based Approaches</h4><p>Deep learning has revolutionized image similarity tasks. Using pre-trained convolutional neural networks (CNNs) like ResNet, VGG, and Inception, you can extract deep features from images.</p><p>CLIP (Contrastive Language-Image Pre-Training) from the openAI is an impressive multimodal zero-shot image classifier that achieves impressive results in a wide range of domains with no fine-tuning. It applies the recent advancements in large-scale transformers like GPT-3 to the vision arena.</p><p>We can fine-tune these models on our own image and text data with the regular SentenceTransformers training code. <a href="https://www.scrapehero.com/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science"><strong>ScrapeHero</strong></a><strong> </strong>helps in preparing your own image dataset to train these models. Its <a href="https://www.scrapehero.com/web-crawling-service/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science">web crawling service</a> can crawl complex websites and provides high-quality data.</p><p>Here, we’ll demonstrate a simple example using CLIP-Based pre-trained model and the <strong><em>torch</em></strong>, <strong><em>open_clip </em></strong>&amp; <strong><em>sentence_transformers </em></strong>libraries:</p><pre>!pip install git+https://github.com/openai/CLIP.git<br>!pip install open_clip_torch<br>!pip install sentence_transformers<br><br>import torch<br>import open_clip<br>import cv2<br>from sentence_transformers import util<br>from PIL import Image<br># image processing model<br>device = &quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;<br>model, _, preprocess = open_clip.create_model_and_transforms(&#39;ViT-B-16-plus-240&#39;, pretrained=&quot;laion400m_e32&quot;)<br>model.to(device)<br>def imageEncoder(img):<br>    img1 = Image.fromarray(img).convert(&#39;RGB&#39;)<br>    img1 = preprocess(img1).unsqueeze(0).to(device)<br>    img1 = model.encode_image(img1)<br>    return img1<br>def generateScore(image1, image2):<br>    test_img = cv2.imread(image1, cv2.IMREAD_UNCHANGED)<br>    data_img = cv2.imread(image2, cv2.IMREAD_UNCHANGED)<br>    img1 = imageEncoder(test_img)<br>    img2 = imageEncoder(data_img)<br>    cos_scores = util.pytorch_cos_sim(img1, img2)<br>    score = round(float(cos_scores[0][0])*100, 2)<br>    return score<br>print(f&quot;similarity Score: &quot;, round(generateScore(image1, image2), 2))<br>#similarity Score: 76.77</pre><p>The similarity between images can then be computed based on the cosine similarity or Euclidean distance of these feature vectors. To improve the accuracy, we can preprocess the images</p><h3>Applications</h3><p>The main applications of the image similarity technique include e-commerce product matching, image retrieval, object recognition, and face recognition. Image similarity, for example, is used in image retrieval to find images similar to a query image.</p><p>Image similarity can be used in object recognition to match a given object with a known database. An image similarity algorithm is used to identify people by comparing their faces to a database.</p><h3>Conclusion</h3><p>The ability to measure image similarity is a vital component of numerous applications in today’s visually driven world. This blog has introduced you to various image similarity approaches, from simple histogram-based methods to complex deep-learning techniques.</p><p>You can also explore the <strong>Siamese networks</strong>, a special class of neural networks designed for one-shot learning and image similarity tasks.</p><p>Python, with its rich ecosystem of libraries like <strong><em>scikit-image</em></strong>,<strong> <em>opencv-python</em></strong>,<strong> <em>TensorFlow</em></strong>, and <strong><em>PyTorch</em></strong>, empowers developers and researchers to implement these approaches effectively.</p><p>Experimenting with these techniques will open doors to creating innovative applications that leverage the power of image similarity.</p><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8ca0a3ed5a3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/exploring-image-similarity-approaches-in-python-b8ca0a3ed5a3">Exploring Image Similarity Approaches in Python</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Streamlining Text Classification Models with MLflow: A Comprehensive Guide]]></title>
            <link>https://medium.com/scrapehero/streamlining-text-classification-models-with-mlflow-a-comprehensive-guide-6cc3ce71ed90?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cc3ce71ed90</guid>
            <category><![CDATA[data-science-courses]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[mlflow]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Sat, 29 Jul 2023 04:49:00 GMT</pubDate>
            <atom:updated>2023-09-01T09:49:27.471Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.scrapehero.com/nlp-basics-abstractive-and-extractive-text-summarization/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science"><strong>Text classification</strong></a> is a fundamental natural language processing (NLP) task that involves categorizing text documents into predefined classes or categories. Machine learning models play a crucial role in solving this problem efficiently. However, developing and managing multiple models can be challenging, especially as projects grow in complexity. This is where <a href="https://mlflow.org/docs/latest/index.html"><strong>MLflow</strong></a><strong> </strong>comes in handy. <strong>MLflow</strong> is an open-source platform that simplifies the end-to-end machine learning lifecycle, making it easier to develop, compare, and deploy text classification models.</p><p>In this blog, we’ll explore the benefits of using MLflow for text classification tasks and guide you through building, tracking, and optimizing your text classification model with MLflow.</p><h3>Why MLflow for Text Classification?</h3><p><strong>MLflow </strong>offers several advantages when it comes to developing and managing text classification models:</p><ol><li><strong>Experiment Tracking</strong>: MLflow allows you to log and track model training runs, hyperparameters, and performance metrics. This helps you keep a comprehensive record of your experiments, making it easier to reproduce and compare results.</li><li><strong>Model Versioning:</strong> With MLflow, you can version control your models, enabling you to manage multiple model iterations and choose the best-performing one for deployment.</li><li><strong>Collaboration:</strong> MLflow promotes seamless collaboration among team members. By sharing MLflow experiments, everyone can access, analyze, and build upon the work done by others.</li><li><strong>Reproducibility:</strong> By maintaining a detailed record of the environment, data, and code used during model training, MLflow facilitates the reproduction of results, even across different systems.</li><li><strong>Deployment Flexibility:</strong> MLflow supports multiple deployment options, making it easier to serve your trained model on different platforms, whether on-premises or in the cloud.</li></ol><h3>Setting up MLflow</h3><p>Before diving into text classification, let’s first set up MLflow in our environment. Ensure you have Python installed, and then use the following commands:</p><pre>pip install mlflow<br>mlflow ui</pre><h3>Building a Text Classification Model</h3><p>For the purpose of this tutorial, we’ll build a text classification model using the news dataset generated by <a href="https://www.scrapehero.com/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science"><strong>ScrapeHero</strong></a>, one of the leading web-scraping companies in the world. <a href="https://www.scrapehero.com/machine-learning-ai-training-data/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=data_science"><strong>ScrapeHero</strong></a> has the capability to scale and crawl the internet for relevant data to help train your AI models. This news dataset is generated from the marketplace API of <a href="https://cloud.scrapehero.com/library/api/api-news/documentation"><strong>ScrapeHero Cloud</strong></a>. This supervised dataset is classified into tennis, politics, crime, entertainment, and business.</p><p>We’ll use <a href="https://github.com/amaiya/ktrain"><strong>ktrain</strong></a><strong> </strong>and scikit-learn for this classification task. Below is the outline of the steps we’ll follow:</p><ul><li>Load and preprocess the dataset. (We will be loading the preprocessed dataset)</li></ul><pre>import pandas as pd<br>try:<br> df = pd.read_csv(&quot;data/sample.csv&quot;)<br> classes = list(set(df.label.tolist()))<br>except Exception as e:<br> logger.exception(<br> &quot;Unable to load training &amp; test CSV, check the file path. Error: %s&quot;, e<br> )</pre><ul><li>Split the data into training, testing, and validation sets.</li></ul><pre>from sklearn import model_selection<br><br>train_x, valid_x, train_y, valid_y = model_selection.train_test_split(df[&#39;content&#39;].tolist(), df[&#39;label&#39;].tolist())<br><br>trn, val, preproc = text.texts_from_array(x_train=train_x, y_train=train_y,<br>                                                x_test=valid_x, y_test=valid_y,<br>                                                class_names=classes,<br>                                                preprocess_mode=&#39;distilbert&#39;,<br>                                                maxlen=256, <br>                                                max_features=10000)</pre><ul><li>Train the text classification model using distilbert.</li></ul><pre>model = text.text_classifier(&#39;distilbert&#39;, train_data=trn, preproc=preproc)<br>learner = ktrain.get_learner(model, train_data=trn, val_data=val, batch_size=6, use_multiprocessing=True)</pre><ul><li>Find the optimal learning rate for our dataset using ktrain inbuilt functions.</li></ul><pre>learner.lr_find(max_epochs=2) # finding the learning rate<br><br># The optimal learning rate is `2e-5`.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*DuZcrrQ_CKwBBaXii42uRQ.png" /><figcaption>Optimal learning rate</figcaption></figure><ul><li>Log the model and relevant parameters like accuracy, f1-score, learning rate, and epochs using MLflow.</li></ul><h3>Tracking with MLflow</h3><p>Once you have your model ready, you can log your experiment using MLflow. This includes logging model parameters, metrics, and artifacts (e.g., saved models, plots). Here’s a code snippet to illustrate the process:</p><pre>import mlflow<br>import mlflow.sklearn<br><br>from sklearn.metrics import accuracy_score, f1_score<br><br>lrate = float(sys.argv[1]) if len(sys.argv) &gt; 1 else 2e-5<br>epochs = int(sys.argv[2]) if len(sys.argv) &gt; 2 else 1<br>print(&quot;training ktrain distilbert model (lrate={:f}, epochs={:f}):&quot;.format(lrate, epochs))<br>mlflow.set_experiment(&quot;text_classification_ktrain&quot;)<br>experiment = mlflow.get_experiment_by_name(&quot;text_classification_ktrain&quot;)<br>with mlflow.start_run(experiment_id=experiment.experiment_id, nested=True):<br>    learner.fit_onecycle(lrate, epochs)<br>    predictor = ktrain.get_predictor(learner.model, preproc)<br>    y_pred = predictor.predict(valid_x)<br>    accuracy = accuracy_score(valid_y, y_pred)<br>    f1 = f1_score(valid_y, y_pred, average=&quot;weighted&quot;)<br>    # logging the metrics and parameters to the mlflow<br>    mlflow.log_metric(&quot;accuracy&quot;, round(accuracy, 4))<br>    mlflow.log_metric(&quot;f1-score&quot;, round(f1, 4))<br>    mlflow.log_param(&quot;lrate&quot;, lrate)<br>    mlflow.log_param(&quot;epochs&quot;, epochs)<br>    # save the model<br>    mlflow.sklearn.log_model(learner, &quot;text_classification1&quot;)</pre><p>you can get the complete working code from this <a href="https://github.com/Vasistareddy/text_classification_ktrain.git"><strong>github repo</strong></a>.</p><p>Run the main file <strong>distilbertTraining.py</strong> using the command:</p><pre>python distilbertTraining.py 2e-6 1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*vz5UkC_4ulnwp5oX2wh0Yg.png" /><figcaption>command line</figcaption></figure><p>Once we successfully run the above command, <strong>mlruns</strong> folder, <strong>experiment Id</strong> folder, <strong>run Id</strong> folder with artifacts, metrics, and params will be automatically created as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*_irBIUpPNqp0BH8GIW8JTQ.png" /><figcaption>folder structure</figcaption></figure><h3>Experiment Comparison and Deployment</h3><p>After logging multiple runs with different models or hyperparameters, you can compare their performance using the MLflow UI. This visual representation helps you make informed decisions about which model to choose for deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*Ot5LzlPzC0DF4lRLBiQCKg.png" /><figcaption>command to run mlflow</figcaption></figure><p>We have trained two models with different learning rates 2e-5 and 2e-6. The learning rate 2e-5 is optimal and produced an accuracy rate of 93.5% whereas the learning rate 2e-6 produced just 63.17%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0k7slZr4URgN_Fsx2SlWCg.gif" /><figcaption>model comparison</figcaption></figure><h3>Find the Best Model Using MLflow Tracking via Code</h3><pre>from mlflow.tracking import MlflowClient<br>import mlflow <br>import numpy as np<br><br># If you called your experiment something else, replace here<br>current_experiment=dict(mlflow.get_experiment_by_name(&quot;text_classification_ktrain&quot;))<br>experiment_id=current_experiment[&#39;experiment_id&#39;]<br><br># To access MLFlow stuff we need to work with MlflowClient<br>client = MlflowClient()<br><br># Searches runs for a specific attribute<br>runs = client.search_runs([experiment_id])<br>print(&quot;total runs&quot;, len(runs), &quot;experiment id&quot;, experiment_id)<br><br># Select the best run according to test_accuracy metric<br>best_run = np.argmax([f.data.metrics[&#39;accuracy&#39;] for f in runs])<br>best_auc = np.round(runs[best_run].data.metrics[&#39;accuracy&#39;], 4)<br>best_runname = runs[best_run].info.run_name<br>best_runid = runs[best_run].info.run_id<br>print(f&quot;Experiment had {len(runs)} rounds&quot;)<br>print(f&quot;Best run name - {best_runname} with run id - {best_runid} has the accuracy of {best_auc}&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nuvy0ns8L0PQCWAbPkj-iQ.png" /><figcaption>Finding the best model</figcaption></figure><p>We can cross-check here</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pyb64yF0ZFQTxDs3huGaVA.png" /><figcaption>MLflow UI</figcaption></figure><p>Other than finding the best model, we can register the model and promote it to production through Python code.</p><p>When it comes to deployment, MLflow offers various options, including serving the model via REST API or integrating it into a web application.</p><p>MLflow documentation has a lot of examples and they can be viewed <a href="https://github.com/mlflow/mlflow/tree/master/examples"><strong>here</strong></a>.</p><p>You can get the complete code of streamlining text classification models with MLflow mentioned above from this <a href="https://github.com/Vasistareddy/text_classification_ktrain.git"><strong>GitHub repo</strong></a>.</p><h3>Conclusion</h3><p>MLflow provides a powerful toolset for building and managing ML models. By leveraging MLflow’s experiment tracking, model versioning, and collaboration features, data scientists and NLP practitioners can streamline their workflow, leading to more efficient and reliable ML models.</p><p>Remember, this blog only scratches the surface of what MLflow can do. As you dive deeper into the world of NLP and text classification, MLflow will undoubtedly become an indispensable part of your machine learning toolkit. Happy streamlining &amp; classifying!</p><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cc3ce71ed90" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/streamlining-text-classification-models-with-mlflow-a-comprehensive-guide-6cc3ce71ed90">Streamlining Text Classification Models with MLflow: A Comprehensive Guide</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analysis of Tesla SuperChargers Locations in the USA]]></title>
            <link>https://medium.com/scrapehero/analysis-of-tesla-superchargers-locations-in-the-usa-ff23159d70ac?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff23159d70ac</guid>
            <category><![CDATA[folium]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Wed, 10 May 2023 06:12:31 GMT</pubDate>
            <atom:updated>2023-09-01T09:52:58.380Z</atom:updated>
            <content:encoded><![CDATA[<p>In recent years, electric cars have become increasingly popular, and one of the most well-known electric car brands is <a href="https://www.tesla.com/"><strong>Tesla</strong></a>. The company’s biggest challenge is to build a robust charging infrastructure to support its vehicles. Tesla has invested heavily in building supercharger stations in strategic locations across the United States to provide easy access for their customers to charge their electric cars.</p><p>In this analysis, we will use <strong>Python</strong> to analyze the distribution of <a href="https://www.scrapehero.com/location-reports/Tesla%20Superchargers-USA/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong>Tesla</strong> <strong>Superchargers</strong> in the United States.</a> We will explore the geographic distribution of the charging stations and identify states and regions with the highest concentration of superchargers.</p><h3>Data Collection</h3><p>We will begin by gathering data on Tesla SuperChargers Locations from <a href="https://www.scrapehero.com/store/product/tesla-superchargers-locations-in-the-usa/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong><em>ScrapeHero</em></strong>.</a> The number of locations available in this dataset is 1,728. This data set was last updated on April 26, 2023. The list of fields provided by the ScrapeHero in the dataset are:</p><blockquote>Store No., Name, Stock Ticker, Provider, Zip Code, URL, Street, Phone, Status, City, Direction URL, Stock Ticker, County, Country, Latitude, Longitude, Updated Date, State</blockquote><h3>Analysis</h3><blockquote>import pandas as pd</blockquote><blockquote>df = pd.read_csv(“Tesla_Superchargers_USA.csv”)</blockquote><blockquote>df.head(n=10)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GaJke62HnMmiuO0fI3Petg.png" /><figcaption>sample dataset</figcaption></figure><h4><strong>Data Visualization</strong></h4><p>To visualize the data, we will use the <a href="https://github.com/python-visualization/folium"><strong>folium</strong></a> library to create an interactive map of the United States. We will plot each supercharger location on the map as a marker, with the size of the marker representing the number of charging stalls at each location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/959/1*xtLug_IQmz1QnEKP905sCg.png" /><figcaption>Map of Tesla Supercharger locations in the USA</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3IdHzYdl1u-81ac2rvwALw.png" /><figcaption>Zoomed in map of Tesla Supercharger locations in the USA</figcaption></figure><h4><strong>Top 10 US States with the most number of supercharger stations</strong></h4><blockquote>import matplotlib.pyplot as plt</blockquote><blockquote>state_counts = df[‘State’].value_counts()[:10]<br>plt.bar(state_counts.index, state_counts.values)<br>plt.show()</blockquote><p>With every state having at least 1 station, California is at the top with 305 stations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*OMq-deHc27o7GlX2EZ5stQ.png" /><figcaption>Bar Chart-State vs Count</figcaption></figure><h4>Representing the state-wise count of stations on Choropleth maps</h4><blockquote>url = (<br> “<a href="https://raw.githubusercontent.com/python-visualization/folium/main/examples/data">https://raw.githubusercontent.com/python-visualization/folium/main/examples/data</a>&quot;<br>)<br>state_geo = f”{url}/us-states.json”</blockquote><blockquote># Create a Folium map centered on the USA<br>usa_map = folium.Map(location=[39.8283, -98.5795], zoom_start=4)</blockquote><blockquote># Group the data by state and count the number of stores in each state<br>store_counts = df.groupby(‘State’)[‘Store No.’].count().reset_index()</blockquote><blockquote># Add a choropleth layer to the map to show the store counts by state<br>folium.Choropleth(<br> geo_data=state_geo,<br> name=’choropleth’,<br> data=store_counts,<br> columns=[‘State’, ‘Store No.’],<br> key_on=’feature.id’,<br> fill_color=’BuPu’,<br> fill_opacity=0.7,<br> line_opacity=0.5,<br> legend_name=’Number of Stores’<br>).add_to(usa_map)</blockquote><blockquote># Display the map<br>usa_map</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gARm4PqZQafS27molRuAVw.png" /><figcaption>Choropleth Map of Tesla Supercharger locations in the USA</figcaption></figure><h4>Scenario: Finding the number of supercharger stations about 100 miles from Yellowstone National Park</h4><blockquote># Importing the geodesic module from the geopy library<br>from geopy.distance import geodesic</blockquote><blockquote># the lat-long data for Yellow_Stone_National_Park<br>Yellow_Stone_National_Park = (44.49780341658212, -110.50204978002547)<br> <br>def findDistance(lat, lon):<br> # Print the distance calculated in km<br> return geodesic(Yellow_Stone_National_Park, (lat, lon)).miles</blockquote><blockquote>df[“distance_from_park”] = df.apply(lambda x:findDistance(x.Latitude, x.Longitude), axis=1)</blockquote><blockquote>df.sort_values([“distance_from_park”])[:10][[“Store No.”, “Name”, “distance_from_park”]]</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*BdVsOXfu9Pvi3IrFcp44vA.png" /><figcaption>Preview</figcaption></figure><p>These 7 stores are around 100 miles from the Yellowstone National Park. West Yellowstone Supercharger is the closest one with 31 miles from the park.</p><h4>Plotting the stations around the Park using Folium</h4><blockquote>df_test = df.sort_values([“Yellow_distance”])[:7][[“Store No.”, “Name”, “distance_from_park”, “Latitude”, “Longitude”]]</blockquote><blockquote># Create a Folium map centered on the USA<br>usa_map = folium.Map(location=[39.8283, -98.5795], zoom_start=4)</blockquote><blockquote># Add markers for each store location<br>for index, row in df_test.iterrows():<br> folium.Marker([row[‘Latitude’], row[‘Longitude’]], popup=row[‘Name’], icon=folium.Icon(icon=”minus”, color=”blue”)).add_to(usa_map)</blockquote><blockquote># Display the map<br>usa_map</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*tj9-p-dcOwfJHZUX-UI4JQ.png" /><figcaption>Stations around the Park</figcaption></figure><p>The data for the analysis is available at <a href="https://www.scrapehero.com/store/product/tesla-superchargers-locations-in-the-usa/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong>ScrapeHero</strong>.</a></p><h4>Conclusion</h4><p>In this blog analysis, we found that the charging stations are distributed across the country, with the highest concentration in California, Texas, and Florida.</p><p>Tesla has invested heavily in building a robust charging infrastructure to support their electric vehicles. With the continued growth of the electric vehicle market, it will be interesting to see how Tesla continues to expand its supercharger network in the future. Thanks for reading!!!</p><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff23159d70ac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/analysis-of-tesla-superchargers-locations-in-the-usa-ff23159d70ac">Analysis of Tesla SuperChargers Locations in the USA</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Report on Starbucks stores co-relating GDP of MSAs of USA]]></title>
            <link>https://medium.com/scrapehero/report-on-starbucks-stores-co-relating-gdp-of-msas-of-usa-d61352f6946c?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/d61352f6946c</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analysis]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[plotting]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Mon, 27 Mar 2023 05:09:03 GMT</pubDate>
            <atom:updated>2025-06-11T06:37:43.690Z</atom:updated>
            <content:encoded><![CDATA[<h3>There are 383 Metropolitan Statistical Areas(MSA) in the USA</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RORLn_aVQIgwrzbSyrSwVA.png" /><figcaption>MSA with GDP(2017) overlayed</figcaption></figure><p>GDP data collected from the <strong>U.S Bureau of Economic Analysis.</strong> The GDP value includes all the industries&#39; revenue for the <strong>year 2017</strong>. <a href="https://apps.bea.gov/iTable/download.cfm"><strong><em>click here</em></strong></a><strong><em> for the GDP dataset.</em></strong></p><h3>There are a total of 14497 Starbucks stores in the USA</h3><h3>6249 stores present in Metropolitan Statistical Areas</h3><p>Download Starbucks’ sample POI data from <a href="https://www.scrapehero.com/store/wp-content/uploads/sxaf72/Starbucks_USA_sample.csv"><strong>ScrapeHero</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gD2WQ-Zoa3vLNk0zu18X0g.png" /><figcaption>Top 10 MSA’s based on GDP along with Starbucks Store Count</figcaption></figure><p><strong>Midland</strong> of <strong>Texas</strong> tops the list <strong><em>174750 million $ GDP </em></strong>but it contains only <strong>6 Starbuck Stores</strong>. The major industry contributor for <strong>Midland, TX </strong>GDP growth is <strong>“Natural resources and mining”.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*rhkLV2kKCp8HunxG5fclCw.png" /><figcaption>source: <a href="https://www.usatoday.com/story/money/business/2016/09/26/cities-fastest-growing-and-shrinking-economies/90920926/">https://www.usatoday.com/story/money/business/2016/09/26/cities-fastest-growing-and-shrinking-economies/90920926/</a></figcaption></figure><p>The MSA <strong>New York-Newark-Jersey City</strong>, NY-NJ-PA is in 10th position with <strong><em>71080 million $ GDP</em></strong> and tops the list with the highest <strong>240 Starbucks stores</strong> count among all the 383 MSAs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3J7pWDJRsztqC-rqDrQ--Q.png" /><figcaption>Top 10 MSA’s based on Starbucks Store Count along with GDP</figcaption></figure><p>The MSA <strong>New York-Newark-Jersey City</strong> has a population of around 20.3 Million with 240 Starbucks stores and while Midland, Tx with the highest GDP has a population of just 170,675.</p><p>There are 18 MSA’s with <strong>Zero</strong> Starbucks stores</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LHLcA5Yq4jJUV4FLwPr7gQ.png" /><figcaption>MSAs with 0 Starbucks stores</figcaption></figure><p>Download the complete dataset of Starbucks POI Data of the USA from <a href="https://www.scrapehero.com/store/product/starbucks-store-locations-in-the-usa/"><strong>ScrapeHero</strong></a>.</p><h3>Fields included in the Starbucks POI Data Source from ScrapeHero are:</h3><ul><li>Store No.</li><li>Zip Code</li><li>Status</li><li>Store Name</li><li>Phone</li><li>Stock Ticker</li><li>Provider</li><li>Open Hours</li><li>NAICS 1</li><li>Street</li><li>Direction URL</li><li>NAICS 2</li><li>City</li><li>Latitude</li><li>SIC 1</li><li>County</li><li>Longitude</li><li>SIC 2</li><li>State</li><li>Stock Ticker</li><li>Updated Date</li><li>Country</li><li>URL</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d61352f6946c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/report-on-starbucks-stores-co-relating-gdp-of-msas-of-usa-d61352f6946c">Report on Starbucks stores co-relating GDP of MSAs of USA</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mississippians are almost “Missing the Sip of Starbucks Coffee”]]></title>
            <link>https://medium.com/scrapehero/mississippians-are-almost-missing-the-sip-of-starbucks-coffee-1c6e310785a2?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c6e310785a2</guid>
            <category><![CDATA[starbucks-coffee]]></category>
            <category><![CDATA[starbucks]]></category>
            <category><![CDATA[poi]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-analysis]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Fri, 24 Mar 2023 05:51:52 GMT</pubDate>
            <atom:updated>2023-09-01T09:53:51.948Z</atom:updated>
            <content:encoded><![CDATA[<p>Starbucks is one of the most popular coffee chains in the world and is present in over 80 countries. Headquartered in the US, there are <strong>14,407 Starbucks stores</strong> across 50 US states.</p><p>Starbucks stores are present in every state of the USA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yHO0eeV-O6zybr9W6ys-_w.png" /><figcaption>Starbucks distribution in the US states</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u1T2rvczhIzeICWiTmNpUg.png" /><figcaption>Top 10 most populous states in the USA</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hyVeyx-n93N36Ae-yDGjDA.png" /><figcaption>population → Starbucks</figcaption></figure><ul><li>California is the most populous state with most Starbucks stores.</li><li>Texas grabs the second position in both population and Store count.</li></ul><p>Interestingly, Washington with 3rd highest number of Starbucks Stores is the 13th most populous state. The reason will be</p><blockquote>The Pike Place Starbucks store, commonly called the Original Starbucks, is the first Starbucks store, established in 1971 at Pike Place Market in downtown Seattle, Washington, United States.</blockquote><h3>Let&#39;s find out who leads in Population to Starbucks store density per state</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O6sw7pRfneo9ph7GwXD5gw.png" /><figcaption>Population to Starbucks store density per state</figcaption></figure><ul><li>One Starbucks store is present for every 9,403 residents in Washington.</li><li>Alaska with just 0.74 Million in population and is the 48th most populous state settled in most love for coffee states.</li><li>Hawaii with just 1.42 Million population, is the 40th most populous state and grabs a place in most love-for-coffee states.</li><li>Only California and Illinois of the top 10 populous states present in most love for coffee states.</li></ul><p><strong>California </strong>and<strong> Hawaii</strong> are the <a href="https://coffeetalk.com/daily-dose/from-origin/05-2022/92296/">only coffee-growing states in the USA</a>. Both secured a spot in most love-for-coffee states.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*efZ8QMi7Q-g_WIq3IXMX9w.png" /><figcaption>density plot</figcaption></figure><ul><li>Mississippians are almost <strong>“Missing the Sip of Starbucks Coffee”</strong> with <strong>just</strong> <strong>one Starbucks store for 90,567 residents</strong>.</li><li><strong>Alabama</strong> with a population of around 5 Million population has 88 Starbucks stores with just one Starbucks store for 55,264 residents.</li></ul><h3><strong>Do you Know?</strong></h3><ul><li><strong>Wyoming</strong> with half a million population(least populous state) contains 23 Starbucks Stores and <strong>Mississippi</strong> with 5 million population contains 33 Starbucks Stores. <strong>Wyoming</strong> with only 10% of the population of <strong>Mississippi</strong> has 10 stores more than Mississippi.</li><li>Seattle in Washington, Portland in Oregon, San Francisco in California, Newyork City in Newyork, Chicago in Illinois, and Denver in Colorado are the coffee-crazed cities from the <a href="https://www.ncausa.org/Newsroom/NCA-releases-Atlas-of-American-Coffee"><strong>National Coffee Association (NCA) survey</strong></a>.</li></ul><p><a href="https://www.scrapehero.com/store/wp-content/uploads/sxaf72/Starbucks_USA_sample.csv"><strong>Download the Starbucks USA locations sample POI data source from ScrapeHero.</strong></a></p><h3>Fields included in the Starbucks store location data source of ScrapeHero are:</h3><ul><li>Store No.</li><li>Zip Code</li><li>Status</li><li>Store Name</li><li>Phone</li><li>Stock Ticker</li><li>Provider</li><li>Open Hours</li><li>NAICS 1</li><li>Street</li><li>Direction URL</li><li>NAICS 2</li><li>City</li><li>Latitude</li><li>SIC 1</li><li>County</li><li>Longitude</li><li>SIC 2</li><li>State</li><li>Stock Ticker</li><li>Updated Date</li><li>Country</li><li>URL</li></ul><p><a href="https://www.scrapehero.com/store/product/starbucks-store-locations-in-the-usa/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong>Download the complete Starbucks POI data from ScrapeHero.</strong></a></p><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c6e310785a2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/mississippians-are-almost-missing-the-sip-of-starbucks-coffee-1c6e310785a2">Mississippians are almost “Missing the Sip of Starbucks Coffee”</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Learning Vs Machine Learning with Text Classification]]></title>
            <link>https://vasista.medium.com/deep-learning-vs-machine-learning-with-text-classification-162ea20a7924?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/162ea20a7924</guid>
            <category><![CDATA[text-classification]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[cnn-model]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Tue, 21 Mar 2023 09:28:08 GMT</pubDate>
            <atom:updated>2023-03-21T09:28:08.322Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wVgSuBIjSBT5lS_5Y-vSHQ.png" /><figcaption>Outline</figcaption></figure><p><strong>Deep Learning</strong>, <strong>Machine Learning</strong>, and <strong>Artificial Intelligence</strong> are the popular buzzwords in present trends.</p><p><strong>Artificial Intelligence</strong> is the branch of computer science that deals with developing intelligence artificially for machines that are able to think, act and behave like humans.</p><p><strong>Machine Learning</strong> is a subset of <strong>Artificial Intelligence</strong> and is the way to implement artificial intelligence. It is the statistical approach where each instance in a data set is described by a set of features or attributes. Feature Extraction is key in <strong>Machine Learning</strong>.</p><p><strong>Deep Learning</strong> is the next evolution and subset of <strong>Machine Learning</strong>. It is a method of statistical learning that extracts features or attributes from raw data. <strong>Deep Learning</strong> uses a network of algorithms called artificial neural networks which imitates the function of the human neural networks present in the brain. <strong>Deep Learning</strong> takes the data into a network of layers(Input, Hidden &amp; Output) to extract the features and to learn from the data.</p><p>In <strong>Machine Learning</strong> and <strong>Deep Learning</strong>, there are models that fall into different categories like supervised, unsupervised &amp; reinforcement learning. In this tutorial, we will discuss Supervised learning which involves an output label associated with each instance in the data set.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*wmWKg_y6jvZU4sg3XhUAZQ.png" /><figcaption>Supervised Learning Model Flow Chart</figcaption></figure><h3>Data Set</h3><blockquote>Data is the new oil</blockquote><p>The above quote says it all. The popularity grew in technologies like deep learning, machine learning and the network of algorithms associated are data-hungry for better efficiency and accuracy. The more data you feed, the Better the accuracy you get by learning.</p><p>For our tutorial, the data set has been taken from <a href="https://www.kaggle.com/uciml/news-aggregator-dataset/version/1?"><strong>Kaggle</strong></a><strong>. </strong>I have chosen 10 of 40 labels from the data set of which 2K values per label. Categorized the data set with the following 10 labels Business, Crime, Entertainment, Food &amp; Drink, Politics, Religion, Science, Sports, Technology, and Travel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LAWM-3GAaJBbrXqqit5cOg.png" /><figcaption>Dataset Overview</figcaption></figure><p>We split the data set into three samples — Train, Validate, and Test samples with percentages 80, 10, and 10 respectively. Train and Validate are for training and validating the model whereas the Test sample is for evaluating the model with F1-Score. The test sample is what the model never encountered while training. We further do analysis with F1-Score of Machine Learning and Deep Learning Models with different data-size and epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*GZEkQnxkv94-Q0PLYaKDlg.png" /></figure><p>The data sets are uploaded <a href="https://github.com/Vasistareddy/text_classification_DL_vs_ML/tree/master/data"><strong>here</strong></a>.</p><h3>Data-Cleaning</h3><p>Pre-processing of data will have an impact on the output ie., the accuracy, and performance of the model. Some of the data-cleaning steps are as follows</p><ul><li>Removing Stop Words. (<strong>NLTK</strong>)</li><li>Performing Stemming on the text. (<strong>NLTK</strong>)</li><li>Removing special characters &amp; extra spaces or keeping only Alpha-Numeric characters in the text</li></ul><pre># NLTK python module for stemming and stopwords removal<br>from nltk.stem.snowball import SnowballStemmer<br>from nltk.corpus import stopwords<br>import string, re</pre><pre>stemmer = SnowballStemmer(&#39;english&#39;) # stemmer<br>t = str.maketrans(dict.fromkeys(string.punctuation)) # special char removal</pre><pre>def clean_text(text):  <br>    ## Remove Punctuation<br>    text = text.translate(t) <br>    text = text.split()</pre><pre>## Remove stop words<br>    stops = set(stopwords.words(&quot;english&quot;))<br>    text = [stemmer.stem(w) for w in text if not w in stops]<br>    <br>    text = &quot; &quot;.join(text)<br>    text = re.sub(&#39; +&#39;,&#39; &#39;, text) # extra consecutive space removal <br>    return text</pre><h3>Goal</h3><p>Our goal is to find the best approach among deep learning and machine learning to perform the classification task. <strong>“Text Classification” </strong>is<strong> </strong>an example of Supervised Machine Learning &amp; Natural Language Processing task.</p><p>The goal of text classification is to automatically classify text documents into one or more predefined categories.</p><h3>Dive into the Deep Learning</h3><p>The fundamental network architectures of neural networks are</p><ol><li><strong>Convolutional Neural Networks (CNN)</strong></li><li><strong>Recurrent Neural Networks (RNN)</strong></li><li><strong>Recursive/Hierarchical Neural Networks (HAN)</strong></li><li><strong>Unsupervised Pre-trained Networks</strong></li></ol><p>For this tutorial, we are focusing only on CNN. CNN shows promising results compared to other architectures, especially for the classification of images/text. Compared to CNN, other architectures are time-consuming too.</p><p><a href="https://en.wikipedia.org/wiki/Convolutional_neural_network"><strong>CNN</strong></a> is a class of deep, feed-forward artificial neural networks &amp; uses a variation of multilayer perceptrons designed to require minimal preprocessing. Here, we use convolutions over the input layer to compute the output instead of the connection of each input neuron to each output neuron in the next layer like a fully connected network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N4h1SgwbWNmtrRhszM9EJg.png" /><figcaption>CNN WorkFlow</figcaption></figure><p>CNNs are generally used in computer vision, however, they’ve recently been applied to various NLP tasks like text classification, sentiment analysis, etc and the results are promising. Unlike image pixels, in our case, a character/word will be the input to the network. NLP tasks contain data which is text/sentence/words represented as a matrix. That is, each row is a vector that represents a word. Typically, these vectors are <a href="https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html"><strong>word embeddings</strong></a> like <a href="https://nlp.stanford.edu/projects/glove/"><strong>Glove</strong></a><strong> </strong>(<em>Global Vectors for Word Representation</em>), word2vec, and TF-IDF, but they could also be one-hot vectors that index the word into a vocabulary. For a 10-word sentence using a 100-dimensional glove embedding, we would have a 10×100 matrix as our input.</p><p>In CNN, we can make convolutions detect a pattern like Ngrams by varying the size of the kernel and concatenating the convolution outputs.</p><p>For example — The sentence is “I like this movie very much”. Convolution kernel filters are [3, 4, 5] so the patterns detected from the sentence will be “I like this”, “I like this movie”, “I like this movie very”, “like this movie”, … etc. which are very useful in the next layers of the network. Let&#39;s have a look at the following diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*2fDin81oI1rXvykUS6M3fg.png" /><figcaption>Source: Google(Altered)</figcaption></figure><pre>convs = []<br>filter_sizes = [3,4,5]<br>MAX_SEQUENCE_LENGTH = 1000</pre><pre>sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype=&#39;int32&#39;)<br>embedded_sequences = embedding_layer(sequence_input)</pre><pre>for filter_size in filter_sizes:<br>    l_conv = Conv1D(filters=128, kernel_size=filter_size, activation=&#39;relu&#39;)(embedded_sequences)<br>    l_pool = MaxPooling1D(5)(l_conv)<br>    convs.append(l_pool)</pre><pre>l_merge = Concatenate(axis=1)(convs)<br>l_cov1= Conv1D(filters=128, kernel_size=5, activation=&#39;relu&#39;)(l_merge)<br>l_cov1 = Dropout(0.2)(l_cov1)<br>l_pool1 = MaxPooling1D(5)(l_cov1)<br>l_cov2 = Conv1D(filters=128, kernel_size=5, activation=&#39;relu&#39;)(l_pool1)<br>l_cov2 = Dropout(0.2)(l_cov2)<br>l_pool2 = MaxPooling1D(30)(l_cov2)<br>l_flat = Flatten()(l_pool2)<br>l_dense = Dense(128, activation=&#39;relu&#39;)(l_flat)<br>preds = Dense(10, activation=&#39;softmax&#39;)(l_dense) # 10 labels = 10 nodes</pre><p>The above Convolutional Architecture uses a total of 128 filters with sizes 3, 4, and 5 and max-pooling of 5 and 30.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*wuIjVSPpm0HrRiLVyEqxgQ.png" /><figcaption>Model Network Architecture</figcaption></figure><p>The final layer dense_2 Output shape will change depending on the data-set categories. Since the data set has 10 categories, the final layer shape is 10.</p><p><strong>Dropout</strong> is dropping off the neurons to prevent an over-fitting problem in neural networks. It is an approach to regularization in neural networks which helps to reduce interdependent learning amongst the neurons. In Machine Learning we use regularization to prevent an over-fitting problem by adding a penalty to the loss function.</p><p><strong>Batch normalization</strong> is another method to regularize a convolutional network.</p><p>How the inclusion of Dropout or/and Batch Normalization affects the performance of the network model will be another topic to discuss.</p><p>The complete code of “<strong>text classification using Deep Learning-CNN</strong>” is <a href="https://github.com/Vasistareddy/text_classification_DL_vs_ML/blob/master/model_training_tutorial_dl.py"><strong>here</strong></a>.</p><h3>Working with the Machine Learning algorithms</h3><p>While in Deep Learning — We know there are different networks and we choose the CNN — Here in Machine Learning, the replacement of network is the algorithm and those algorithms are:</p><ul><li>Naive Bayes.</li><li>Decision Trees.</li><li>Logistic Regression <em>(Linear Model)</em>.</li><li>Support Vector Machines <em>(SVM)</em>.</li><li>Random Forest.</li><li>K-Means Clustering.</li><li>K-Nearest Neighbour.</li><li>Gaussian Mixture Model.</li><li>Hidden Markov Model. <strong>et cetera</strong></li></ul><p>Among these ML Algorithms, we will discuss how the <strong>Naive Bayes</strong>, <strong>Logistic Regression</strong> and <strong>SVM</strong> classifier models perform on the data-set feature vectors.</p><p>Transforming the data into feature vectors with the following methods</p><ul><li>Count Vectors.</li><li>TF-IDF Word Vectors.</li><li>TF-IDF N-Gram Vectors.</li></ul><pre>from sklearn import model_selection, preprocessing<br>from sklearn.feature_extraction.text import TfidfVectorizer, CountVectorizer</pre><pre>&#39;&#39;&#39;Assume df is the dataset with columns &quot;content&quot; and &quot;label&quot;&#39;&#39;&#39;</pre><pre># split the data into training and validation<br>train_x, valid_x, train_y, valid_y = model_selection.train_test_split(df[&#39;content&#39;], df[&#39;label&#39;])</pre><pre># label encode the target variable <br>encoder = preprocessing.LabelEncoder()<br>train_y = encoder.fit_transform(train_y)<br>valid_y = encoder.fit_transform(valid_y)</pre><pre>target_names = list(encoder.classes_) # output labels for report generation</pre><pre># count vectorization <br>count_vect = CountVectorizer(analyzer=&#39;word&#39;, token_pattern=r&#39;\w{1,}&#39;)<br>count_vect.fit(df[&#39;content&#39;])<br>xtrain_count = count_vect.transform(train_x)<br>xvalid_count = count_vect.transform(valid_x)</pre><pre># word level tf-idf vectorization<br>tfidf_vect = TfidfVectorizer(analyzer=&#39;word&#39;, token_pattern=r&#39;\w{1,}&#39;, max_features=5000)<br>tfidf_vect.fit(df[&#39;content&#39;])<br>xtrain_tfidf = tfidf_vect.transform(train_x)<br>xvalid_tfidf = tfidf_vect.transform(valid_x)</pre><pre># ngram level tf-idf vectorization<br>tfidf_vect_ngram = TfidfVectorizer(analyzer=&#39;word&#39;, token_pattern=r&#39;\w{1,}&#39;, ngram_range=(2,3), max_features=5000)<br>tfidf_vect_ngram.fit(df[&#39;content&#39;])<br>xtrain_tfidf_ngram = tfidf_vect_ngram.transform(train_x)<br>xvalid_tfidf_ngram = tfidf_vect_ngram.transform(valid_x)</pre><p>We use Naive Bayes, Logistic Regression, and SVM algorithms to train the data-set feature vectors to form classifier models that are used for prediction.</p><pre><strong>from sklearn import naive_bayes, metrics<br>from sklearn.metrics import classification_report</strong></pre><pre><strong>def</strong> <strong>report_generation</strong>(classifier, train_data, valid_data, train_y, valid_y):<br>   classifier.fit(train_data, train_y)<br>   predictions = classifier.predict(valid_data)<br>   print(&quot;Accuracy :&quot;, metrics.accuracy_score(predictions, valid_y))<br>   report = classification_report(valid_y, predictions, output_dict=True, target_names=target_names)<br>   <strong>return report</strong></pre><pre><strong># Naive Bayes</strong><br>classifier = <strong>naive_bayes.MultinomialNB()</strong><br>report = report_generation(classifier, xtrain_count, xvalid_count, train_y, valid_y)<br><strong>print(&quot;NB Count Vectorizer Report :&quot;, report[&#39;weighted avg&#39;])</strong></pre><pre># <strong>Results</strong><br><strong>Accuracy</strong> : 0.7948773138183384<br><strong>NB Count Vectorizer Report</strong> : {&#39;precision&#39;: 0.7967624232183244, &#39;recall&#39;: 0.7948773138183384, &#39;f1-score&#39;: 0.7941004411892886, &#39;support&#39;: 4646}</pre><p>Similarly, <strong>Logistic Regression</strong> and <strong>SVM</strong> classifiers are passed to get the <strong>classification report</strong>.</p><pre><strong>from sklearn import linear_model, naive_bayes, svm, metrics</strong><br><strong>from sklearn.metrics import classification_report</strong></pre><pre><strong>def</strong> <strong>report_generation</strong>(classifier, train_data, valid_data, train_y, valid_y):<br>   classifier.fit(train_data, train_y)<br>   predictions = classifier.predict(valid_data)<br>   print(&quot;Accuracy :&quot;, metrics.accuracy_score(predictions, valid_y))<br>   report = classification_report(valid_y, predictions, output_dict=True, target_names=target_names)<br>   <strong>return report</strong></pre><pre><strong># Naive Bayes</strong><br>classifier = <strong>naive_bayes.MultinomialNB()</strong><br>report = report_generation(classifier, xtrain_count, xvalid_count, train_y, valid_y)<br><strong>print(&quot;NB Count Vectorizer Report :&quot;, report[&#39;weighted avg&#39;])</strong></pre><pre><strong># Logistic Regression </strong>   </pre><pre>classifier = <strong>linear_model.LogisticRegression()</strong>    <br>report = report_generation(classifier, xtrain_count, xvalid_count, train_y, valid_y)    <br><strong>print(&quot;LogisticRegression Count Vectorizer Report :&quot;, report[&#39;weighted avg&#39;])</strong><br>  <br>report = report_generation(classifier, xtrain_tfidf, xvalid_tfidf, train_y, valid_y)    <br><strong>print(&quot;LogisticRegression TFIDF-Word Report :&quot;, report[&#39;weighted avg&#39;])</strong>     </pre><pre>report = report_generation(classifier, xtrain_tfidf_ngram, xvalid_tfidf_ngram, train_y, valid_y)    <br><strong>print(&quot;LogisticRegression TFIDF-NGram Report :&quot;, report[&#39;weighted avg&#39;])</strong></pre><pre><strong># Support Vector Machines</strong><br>    <br>classifier = <strong>svm.SVC(gamma=&quot;scale&quot;)</strong>    <br>report = report_generation(classifier, xtrain_count, xvalid_count, train_y, valid_y)    <br><strong>print(&quot;SVM Count Vectorizer Report :&quot;, report[&#39;weighted avg&#39;])</strong></pre><pre><strong># Results</strong><br><strong>Accuracy</strong> : 0.9668<br><strong>SVM Count Vectorizer Report</strong> : {&#39;precision&#39;: 0.9687847838287942, &#39;recall&#39;: 0.9668, &#39;f1-score&#39;: 0.9672306318670637, &#39;support&#39;: 2500}</pre><pre>report = report_generation(classifier, xtrain_tfidf, xvalid_tfidf, train_y, valid_y)    <br><strong>print(&quot;SVM TFIDF-Word Report :&quot;, report[&#39;weighted avg&#39;])</strong> </pre><pre><strong># Results<br>Accuracy</strong> : 0.9804<br><strong>SVM TFIDF-Word Report</strong> : {&#39;precision&#39;: 0.980766234757573, &#39;recall&#39;: 0.9804, &#39;f1-score&#39;: 0.9804795388691244, &#39;support&#39;: 2500}<br>    <br>report = report_generation(classifier, xtrain_tfidf_ngram, xvalid_tfidf_ngram, train_y, valid_y)    <br><strong>print(&quot;SVM TFIDF-NGram Report :&quot;, report[&#39;weighted avg&#39;])</strong></pre><pre><strong># Results</strong><br><strong>Accuracy</strong> : 0.9304<br><strong>SVM TFIDF-NGram Report</strong> : {&#39;precision&#39;: 0.9324797933370057, &#39;recall&#39;: 0.9304, &#39;f1-score&#39;: 0.9306949638900389, &#39;support&#39;: 2500}</pre><p>The complete code of “<strong>text classification using Machine Learning-Naive Bayes, Logistic Regression and SVM</strong>” is <a href="https://github.com/Vasistareddy/text_classification_DL_vs_ML/blob/master/model_training_tutorial_ml.py"><strong>here</strong></a>.</p><h3>Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*aCdv76w8YxE5JswyqeLcyw.png" /></figure><h3>Observations</h3><ul><li>CNN outperformed the ML Algorithms like Naive Bayes, SVM, and Logistic Regression in all the 3 cases. Neural Networks showed tremendous accuracy for the NLP text classification task.</li><li>Among the ML algorithms, Naive Bayes with Count Vectorization has better accuracy.</li><li>An increase in the number of epochs increased the accuracy in the case of CNN of the respective data set.</li><li>More Epochs = Better Accuracy of the model considering the sample size.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=162ea20a7924" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[McDonald's Stores Analysis of Canada]]></title>
            <link>https://medium.com/scrapehero/mcdonalds-stores-analysis-of-canada-a4e3b190243c?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/a4e3b190243c</guid>
            <category><![CDATA[poi]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[mcdonalds]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Tue, 21 Mar 2023 09:23:49 GMT</pubDate>
            <atom:updated>2023-09-01T09:54:40.628Z</atom:updated>
            <content:encoded><![CDATA[<p>McDonald’s is one of the world&#39;s most popular fast-food chains. <strong>Western Franchise Owners</strong> opened the first McDonald’s restaurant in Canada in 1967 in Richmond, British Columbia. It was also the <a href="https://www.mcdonalds.com/ca/en-ca/about-us/our-history.html">first McDonald’s restaurant outside of the United States</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vf1UMSTYo9LaLMhM5Vr6PQ.png" /></figure><p>With a total of <a href="https://www.scrapehero.com/location-reports/McDonalds-Canada/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong>1,396 McDonald&#39;s stores</strong> in Canada</a>, <strong>Ontario state</strong> leads with<strong> 504 stores </strong>i.e., <strong>36% </strong>of the total locations in Canada.</p><h3>Locating the closest McDonald’s Stores to the tourist attractions in Canada</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*XwAKgaOhvcvPRC9czYEq1w.gif" /><figcaption>Network graph of McDonald’s near Niagra Falls</figcaption></figure><p>From <strong>Niagara Falls — “5486 STANLEY AVENUE, NIAGARA FALLS, Canada, L2G 3X2” </strong>is the closest McDonald’s store with around <strong>0.5 Miles</strong> to <strong>Niagara Falls Canada.</strong></p><p>We have <strong>5 stores</strong> around <strong>5 miles</strong> from <strong>Niagara Falls.</strong> The next store is around<strong> 11 miles.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vCDlcBcojtEzfMb3YizNGg.png" /></figure><p><a href="https://www.scrapehero.com/store/wp-content/uploads/sxaf72/McDonalds_Canada_sample.csv"><strong>For custom analysis, download the sample POI data from ScrapeHero.</strong></a></p><h3>Fields included in the McDonald’s store location data source of ScrapeHero are:</h3><ul><li>Store No.</li><li>Zip Code</li><li>Stock Ticker</li><li>Store Name</li><li>Phone</li><li>URL</li><li>Provider</li><li>Open Hours</li><li>Status</li><li>Street</li><li>Direction URL</li><li>Stock Ticker</li><li>City</li><li>Latitude</li><li>Updated Date</li><li>Country</li><li>Longitude</li><li>Province</li></ul><p><a href="https://www.scrapehero.com/store/product/mcdonalds-store-locations-in-canada/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=location_data"><strong>Download the complete POI data source from ScrapeHero.</strong></a></p><h3>Did you know?</h3><ul><li><a href="https://en.wikipedia.org/wiki/McBarge"><strong>McBarge</strong></a>(<strong><em>Friendship 500)</em></strong>, was a McDonald’s restaurant, built on a 187-foot-long (57 m) barge for Expo ’86 in <strong>Vancouver, British Columbia, Canada</strong>. It is the world&#39;s second abandoned floating McDonald’s restaurant (the first being in <a href="https://en.wikipedia.org/wiki/St_Louis,_Missouri">St Louis, Missouri</a>), intended to showcase future technology and architecture. — source: Wiki</li><li>In 2003, <a href="https://en.wikipedia.org/wiki/Marvel_Entertainment">Marvel Entertainment</a> and <a href="https://en.wikipedia.org/wiki/New_Line_Cinema">New Line Cinema</a> rented the<a href="https://en.wikipedia.org/wiki/McBarge"> Mcbarge </a>as a filming location for the 2004 film <a href="https://en.wikipedia.org/wiki/Blade:_Trinity"><em>Blade: Trinity</em></a> as the lair of the <a href="https://en.wikipedia.org/wiki/Nightstalkers_(comics)">Nightstalkers</a>. — source: Wiki</li></ul><p><em>If you’ve found this article helpful or intriguing, don’t hesitate to give it a clap! As a writer, your feedback helps me understand what resonates with my readers.</em></p><p><em>Follow </em><a href="https://medium.com/scrapehero"><em>ScrapeHero</em></a><em> for more insightful content like this. Whether you’re a developer, an entrepreneur, or someone interested in web scraping, machine learning, AI, etc., ScrapeHero has compelling articles that will fascinate you.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4e3b190243c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scrapehero/mcdonalds-stores-analysis-of-canada-a4e3b190243c">McDonald&#39;s Stores Analysis of Canada</a> was originally published in <a href="https://medium.com/scrapehero">ScrapeHero</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solo trip to Chikmagalur from Bangalore on Suzuki Access 125cc]]></title>
            <link>https://vasista.medium.com/solo-trip-to-chikmagalur-from-bangalore-on-suzuki-access-125cc-e4dc6642110a?source=rss-6bbf0e698487------2</link>
            <guid isPermaLink="false">https://medium.com/p/e4dc6642110a</guid>
            <category><![CDATA[tourism]]></category>
            <category><![CDATA[solo-travel]]></category>
            <category><![CDATA[bengaluru]]></category>
            <category><![CDATA[chikmagalur]]></category>
            <category><![CDATA[trekking]]></category>
            <dc:creator><![CDATA[Vasista Reddy]]></dc:creator>
            <pubDate>Fri, 12 Nov 2021 11:10:19 GMT</pubDate>
            <atom:updated>2021-11-12T11:10:19.458Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yCjROpskFNzcCNt82olbwQ.jpeg" /></figure><h3>Day 1: Bangalore to Chikmagalur</h3><p>Started on the weekend ie., Saturday morning at 5:26 AM on my Suzuki Access 125 CC towards Chikmagalur. Planned to cover temples on the go before reaching Chikmagalur.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FMbo9VciqsL7W5XH_Gqefg.jpeg" /><figcaption>Day 1 Route Map</figcaption></figure><p>The first destination is Shravanabelagola temple which is at 170 Kms from the start. You will encounter a 58.8-foot high monolithic Gommateshwara statue(bahubali) on the top of the hill.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*duFWWgYRWOqJgrALRUzGxQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UdhjsUIsHVuGwtjDFI6m8w.jpeg" /></figure><p>The second destination on Day 1 is Halebeedu <a href="https://en.wikipedia.org/wiki/Hoysaleswara_Temple">Hoysaleswara Temple</a>. The impeccable stunning architecture surely surprises you. Hire a guide to know better about the temple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qHJkRDrwMnOdqPq3Qnhmyw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CX0h8wofa-ccWPCMjt0HMg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BGV1bcRjRWTL5duPPxlHjg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*94wN5PeLTHhqmNDbg4ZEuw.jpeg" /><figcaption><a href="https://en.wikipedia.org/wiki/Hoysaleswara_Temple">Hoysaleswara Temple</a></figcaption></figure><p>The third destination of Day 1 is Belur Chennakeshava Temple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Uwj7492G2ZXFKzxfg89aTQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BR2vHX14aWWKnDiWNaBtOg.jpeg" /><figcaption>Chennakeshava Temple</figcaption></figure><p>Reached Chikmagalur at 2:30 PM. The weather was welcoming throughout the journey.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8i4AFP6cfXpPitcZ4ReftQ.jpeg" /></figure><p>Checked-in a hotel and relaxed for a couple of hours. I am not here to relax right? Went to the man-made Hirekolale lake in the evening at 5:30 which is around 12 Kms from my stay</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QUePIj3l5Dg2dQeJU6CiDg.jpeg" /><figcaption>Hirekolale Lake</figcaption></figure><p>Came back to my stay and ordered biryani from Taj hotel which was good after the tiresome day.</p><h3>Day 2: Sitalayyanagari, Mullayangiri, Baba Budangiri, Siri Coffee, Town Canteen, MG Park</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*22OxM24V190wxFtc1uAF9A.jpeg" /><figcaption>Day 2 Route Map</figcaption></figure><p>Started 6 AM in the morning to Sitalayyanagiri and Mullayangiri Peak.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vNahqEEpGIcDK_Xf2LGZUA.jpeg" /><figcaption>Mullayangiri</figcaption></figure><p>Went on to ride on my scooter till Baba Budangiri.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HRCVBKN-zXPBD4Vk_yZcHA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n1swhAhWNWCWXsuogsjC0Q.jpeg" /></figure><p>On my way back went to Siri Coffee at around 2 PM and the famous Town Canteen for having butter masala dosa which tasted just Okay.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ss7cx1dhF9s1k-y4HpphWA.jpeg" /><figcaption>Mother Nature at Siri Coffee</figcaption></figure><p>After the lunch at Town Canteen, went back to my stay to relax for a couple of hours. At 5 PM, started to explore the town. Went to MG Children&#39;s park</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5876VcLHpPukmngyKci5A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2ncc0ekrjpBVVDNixbYyvg.jpeg" /><figcaption>Mahatma Gandhi Park</figcaption></figure><p>MG Park is pure entertainment for the kids and lovers — others avoid it.</p><p>After a quick look at the entire park, I decided to go to Hirekolale lake for the second time to feel the atmosphere around the lake. It is so pleasant and calm.</p><p>While returning went to RR Donne Biryani, near Bus Station — took the CB parcel and returned to the stay. It was just okay (3/5).</p><h3>Day 3: Coffee Museum and an unmotivated return trip to the nest</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Hvp89L-7rky6h4vqY2GUQ.jpeg" /><figcaption>Day 3 Route Map</figcaption></figure><p>Coffee Museum - strangely closed on Sat and Sun and it opens 10 AM from Mon to Friday.</p><p>Woke up late at 9 AM and got ready to visit Coffee Museum. I was there at the coffee museum exactly at 9:57 AM after having Idly, Vada and Sambar at Town Canteen. I am the only visitor and no employee has arrived at Coffee Museum. After waiting for 30 mins, an employee has come to give directions to look around the museum. It looks like a central Government job and no one bothers to attract visitors. The museum is very neat, good, and informative.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eofFWFWnLEoRP-PRLQrqGA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*L7kLQ_kQ96MrPvlzBjM5hw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*XXtTdsnE_Bkcz5Sq18VoAw.jpeg" /><figcaption>coffee museum</figcaption></figure><p>Back to my stay and started to Bengaluru by 11:30 AM. Return journeys are really painful and unmotivated. From Nelamangala, rain has stopped. I was prepared for it. Reach home by 5:30 PM.</p><h3>Miscellaneous Information</h3><blockquote>Travelled around 700 Kms approximately</blockquote><blockquote>Filled 1500/- rupees petrol</blockquote><blockquote>CB at Hotel Taj was very good(4/5)</blockquote><blockquote>Butter Masala Dosa was Average at Town Canteen(3/5)</blockquote><blockquote>CB at Donne Biryani was Okay(3/5)</blockquote><blockquote>Idly, Vada and Sambar was not great at Town Canteen — Their service is nice but tastes of chutney and sambar are not great.(2/5)</blockquote><blockquote>The hotel price was cheap. Around 1200/- for 2 nights. But the homestay near mullayangiri will be exciting. I saw a lot of homestays around that area.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e4dc6642110a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>