<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tattooed Geek on Medium]]></title>
        <description><![CDATA[Stories by Tattooed Geek on Medium]]></description>
        <link>https://medium.com/@anishsinghwalia?source=rss-3bfe9db19781------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*WczX2msZ4nz1fCDCBJPZFg.png</url>
            <title>Stories by Tattooed Geek on Medium</title>
            <link>https://medium.com/@anishsinghwalia?source=rss-3bfe9db19781------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:41:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anishsinghwalia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[VeLO: The Intelligent Neural Network Optimizer Revolutionizing Deep Learning and Automating…]]></title>
            <link>https://anishsinghwalia.medium.com/velo-the-intelligent-neural-network-optimizer-revolutionizing-deep-learning-and-automating-b8c2c2d93da2?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8c2c2d93da2</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[love]]></category>
            <category><![CDATA[google-research]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Thu, 27 Jul 2023 04:33:14 GMT</pubDate>
            <atom:updated>2024-01-11T13:24:04.603Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>VeLO: The Intelligent Neural Network Optimizer Revolutionizing Deep Learning and Automating Optimization</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2qJWsIWdDQrjIhB0b39EgA.png" /><figcaption>In the proposed method, a new optimizer network takes the target network’s gradients, weights, and current training step and outputs its weight updates — no hyperparameters are needed. | Designed in Canva by <a href="https://medium.com/u/3bfe9db19781">Anish Singh Walia</a></figcaption></figure><p><a href="https://arxiv.org/abs/2211.09760?utm_campaign=The%20Batch&amp;utm_source=hs_email&amp;utm_medium=email&amp;_hsenc=p2ANqtz-98Z0EEpyu-3WG5zBBBGen28QtQATna5cMllWRxO7eBqQ-IMiqGxR4sZnxp2nFzIAc70bQt"><strong>VeLO</strong></a><strong>, is a system designed to act as a fully tuned optimizer. It uses a neural network to compute the target network’s updates — A neural network to train and auto-tune another neural network.</strong></p><blockquote><strong>NOTE:</strong> To add more value and as a USP of my blog, at the end of this blog post, I have designed and attached a cheat sheet/carousel of the topic discussed here for you to use and share.</blockquote><blockquote>I’d love to share the high-quality PDF version of my blog’s cheatsheets with you — for free! Since Medium doesn’t allow uploading PDFs, I post them daily on my <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> page. Let’s connect and stay updated @ <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">https://www.linkedin.com/in/anish-singh-walia-924529103/</a></blockquote><blockquote>Also, I’ll share this month’s bonus tip or best productivity tools that are cheap, effective, and a game changer, which I personally use, prefer, and insist you all try. So do check them out and use them.</blockquote><p>Here is the Bonus tip for you all:</p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><blockquote>I really insist you to go try the above tools out. Trust me, you won’t regret using these tools and will thank me later.</blockquote><h3>INDEX</h3><ol><li><strong>Introduction</strong></li><li><strong>The Struggle of Hyperparameter Tuning</strong></li><li><strong>Critical Insight: Simplifying Hyperparameter Tuning</strong></li><li><strong>How VeLO Works: The Optimizer Network-An Evolutionary Approach</strong></li><li><strong>Diverse Training Scenarios for VeLO</strong></li><li><strong>Implementing VeLO in Python</strong></li><li><strong>Results: Faster Training, Lower Loss</strong></li><li><strong>The Limitations and the Road Ahead</strong></li><li><strong>Why VeLO Matters: Simplifying Model Development</strong></li><li><strong>The Future of Optimizers: VeLO-Like Algorithms</strong></li><li><strong>FAQs — Answering Your VeLO Queries</strong></li><li><strong>References</strong></li></ol><h3><strong>1. Introduction:</strong></h3><p>Training neural networks is a complex process involving optimizers updating their weights. However, finding the best hyperparameters for these optimizers often requires trial and error, making it time-consuming and cumbersome. But fear not!<a href="https://arxiv.org/abs/2211.09760"> <strong>Google’s VeLO (Learned Optimizer)</strong></a> is here to revolutionize the game.</p><blockquote><strong>VeLO acts as a fully tuned optimizer, using a neural network to compute updates for the target network, and the best part is it eliminates the need for hyperparameter tuning!</strong></blockquote><blockquote>Let’s dive into how VeLO works and why it’s a game-changer.</blockquote><p>Training a neural network is like tuning an instrument to produce the perfect melody. Optimizers play a vital role in fine-tuning the network’s weights in AI and deep learning models. What if we could eliminate the need for manually selecting hyperparameters?</p><p>That’s where <a href="https://github.com/google/learned_optimization/tree/main">VeLO (Versatile Learned Optimizer)</a> steps in. Developed by Luke Metz, James Harrison, and their team at Google, <a href="https://arxiv.org/abs/2211.09760">VeLO</a> acts as a fully <strong>tuned optimizer, using a neural network to compute the updates for the target network</strong>. Let’s dive into this groundbreaking system and understand how it redefines <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c"><strong>deep learning optimization</strong></a>.</p><blockquote>In-depth dive to learn more about conventional Optimization algorithms like <strong><em>Adam, Gradient Descent, AdaDelta</em></strong> can be found on my previous blog post below. I urge the readers to go through these conventional optimization techniques to understand the significant differences between VeLO and other optimizers for deep neural networks.</blockquote><p><a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">Types of Optimization Algorithms used in Neural Networks and Ways to Optimize Gradient Descent</a></p><h3>2. The Struggle of Hyperparameter Tuning:</h3><p>For machine learning engineers, hyperparameter tuning can be frustrating and time-consuming. Finding the right combination of <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">optimizer</a> hyperparameters, suchThe Struggle of Hyperparameter Tuning: as learning rate, schedule, and weight decay, often involves hit-and-trial experiments. <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">Traditional optimizers</a> require human expertise to hand-pick hyperparameters like learning rate, schedule, and weight decay. This trial-and-error process is cumbersome, as it involves repeatedly training the target network with different hyperparameter values.</p><p>But VeLO is here to redefine optimization, making it hassle-free. VeLO proposes a unique solution that eliminates the need for such manual tuning.</p><h3><strong>3. Critical Insight: Simplifying Hyperparameter Tuning</strong></h3><p><strong>The genius behind VeLO lies in its ability to automate the optimization process without relying on hyperparameter tuning</strong>. Instead of using traditional optimization techniques like SGD or Adam, <em>VeLO introduces a different neural network, the </em><strong><em>optimizer network</em></strong><em>, to compute weight updates for the target network. This novel approach revolutionizes the way we optimize neural networks.</em></p><p>Hyperparameter tuning is a challenge for machine learning engineers, as finding the best values for optimizer hyperparameters, like learning rate, schedule, and weight decay, can be tedious. The proposed VeLO approach simplifies this by using a separate neural network to compute the weight updates, eliminating the need for manual hyperparameter tuning.</p><h3><strong>4. How VeLO Works: The Optimizer Network-</strong>An Evolutionary Approach</h3><p>At each training step of the target network, VeLO employs an <a href="https://www.analyticsvidhya.com/blog/2021/03/introduction-to-long-short-term-memory-lstm/">LSTM</a> (Long Short-Term Memory) to generate<strong> weights</strong> for what we’ll call the <strong>optimizer network — a vanilla neural network</strong> responsible for updating the <strong>target network</strong>. <strong>The clever part lies in how the LSTM learns to generate the optimizer network’s weights.</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFpveV9O00%2Fview%3Fembed%26meta&amp;display_name=Canva&amp;url=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFpveV9O00%2Fwatch&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=canva" width="2240" height="1260" frameborder="0" scrolling="no"><a href="https://medium.com/media/a6e9870b8418204b6c02b6c9c10adc16/href">https://medium.com/media/a6e9870b8418204b6c02b6c9c10adc16/href</a></iframe><p><strong>A </strong><a href="https://blog.insightdatascience.com/a-quick-introduction-to-vanilla-neural-networks-b0998c6216a1"><strong>Vanilla Neural</strong></a><strong> network is a traditional neural network model with an input, hidden, and output layer.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jpskk9_AjRroOyJ1Lj4Isg.png" /><figcaption>Vanilla Neural Network | Created in Canva by <a href="https://medium.com/u/3bfe9db19781">Anish Singh Walia</a></figcaption></figure><p>Instead of using <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">backpropagation</a>, the <a href="https://www.analyticsvidhya.com/blog/2021/03/introduction-to-long-short-term-memory-lstm/">LSTM</a> evolves through an iterative process. Multiple LSTMs with random differences are generated, and the best ones are <strong>averaged</strong>.<em> New LSTMs closest to the </em><strong><em>average</em></strong><em> are generated, and this process continues. The LSTM is learning from its own “</em><strong><em>experience</em></strong><em>” to improve its </em><strong><em>optimizer network weights</em></strong><em>.</em></p><blockquote>The LSTM doesn’t learn its weights through the typical backpropagation method but leverages an evolution-like process.</blockquote><h3><strong>5. Diverse Training Scenarios for VeLO</strong></h3><p>To make VeLO versatile, the authors randomly generated around <strong>100,000</strong> target neural networks with various architectures —<strong> vanilla, convolutional, recurrent, transformers, etc</strong>. These networks were trained on tasks ranging from image classification to text generation.</p><blockquote>Each LSTM, starting with random weights, was copied and randomly modified for each target network. The modified LSTMs generated <strong>weights</strong> for the optimizer network based on various statistics of the target network. These statistics included mean and variance of weights, moving averages of gradients during training, fraction of completed training steps, and training loss values.</blockquote><h3>6. Implementing VeLO in Python:</h3><p>You can explore the Python SDK <a href="https://github.com/anishsingh20/learned_optimization/tree/main/learned_optimization/research/general_lopt">learned_optimization</a> developed by the researchers to implement VeLO. You can try out the full implementation in Google Colab using the link below:</p><p><a href="https://github.com/anishsingh20/learned_optimization/tree/main/learned_optimization/research/general_lopt">learned_optimization/learned_optimization/research/general_lopt at main · anishsingh20/learned_optimization</a></p><p><a href="https://colab.research.google.com/github/google/learned_optimization/blob/main/learned_optimization/research/general_lopt/Demo_for_training_a_model_with_a_learned_optimizer.ipynb#scrollTo=RQBACAPQZyB-">VeLO Python Code Implementation</a></p><p><a href="https://colab.research.google.com/github/google/learned_optimization/blob/main/learned_optimization/research/general_lopt/Demo_for_training_a_model_with_a_learned_optimizer.ipynb#scrollTo=RQBACAPQZyB-">Google Colaboratory</a></p><p>By following this example, you can seamlessly integrate VeLO as the optimizer for your neural network, eliminating the hassle of hyperparameter tuning.</p><h3><strong>7. Results: Faster Training, Lower Loss</strong></h3><p>VeLO’s performance was evaluated using 83 tasks on a dataset scaled to complete training on a single GPU in one hour. The results were impressive. VeLO trained networks faster than <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">Adam</a>, a famous optimizer, tuned to find the best learning rate.<strong><em> On half of the tasks, VeLO was four times faster than Adam</em></strong>. Additionally, VeLO achieved a <strong>lower loss </strong>than <a href="https://medium.com/nerd-for-tech/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c">Adam</a> on five out of six MLCommons tasks, including image classification, speech recognition, text translation, and graph classification tasks.</p><h3><strong>8. The Limitations and the Road Ahead</strong></h3><p>Though VeLO proved exceptional for most tasks, it underperformed with large models (<strong>over 500 million parameters</strong>) and long training runs (<strong>more than 200,000 steps</strong>). The authors believe training VeLO on larger networks and for more steps(<em>as large as the ones mentioned in the bold above</em>), could overcome this limitation.</p><h3><strong>9. Why VeLO Matters: Simplifying Model Development</strong></h3><p>VeLO accelerates model development by eliminating hyperparameter testing and speeding up optimization. It takes advantage of various statistics from the target network’s training, allowing it to compute updates that move models closer to better solutions.</p><p>The field of optimizers is on the verge of a revolution. As VeLO paves the way for more intelligent optimization, we eagerly anticipate more advanced variants that can handle larger architectures and extended training durations.</p><h3>10. The Future of Optimizers: VeLO-Like Algorithms:</h3><p>VeLO represents a significant leap in deep learning optimization. While it may have room to grow with larger architectures, it has already proven its worth by automating the optimization process and outperforming traditional optimizers in various tasks. As the AI community looks forward to more advanced variants of VeLO, we can confidently say that optimizers are becoming more intelligent, freeing up valuable time and effort for machine learning engineers.</p><h3><strong>11. FAQs — Answering Your VeLO Queries:</strong></h3><ol><li><strong>What sets VeLO apart from traditional optimizers?</strong> VeLO eliminates the need for hyperparameter tuning, automatically adapting to tasks without manual intervention.</li><li><strong>Is VeLO compatible with popular deep-learning frameworks?</strong> Yes, VeLO can be implemented using frameworks like TensorFlow and PyTorch.</li><li><strong>What are the key benefits of using VeLO</strong>? VeLO accelerates model development by automating hyperparameter tuning and optimizing neural networks efficiently.</li><li><strong>Can VeLO handle large models and extended training durations?</strong> VeLO’s performance might be limited to extensive models and vast training, but further research can address this.</li></ol><h3><strong>12. References:</strong></h3><ol><li>Research Paper: [<a href="https://arxiv.org/abs/2211.09760">VeLO: Training Versatile Learned Optimizers by Scaling Up</a>] Authors: Luke Metz, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal, Ben Poole, Igor Mordatch, Adam Roberts, Jascha Sohl-Dickstein</li><li><a href="https://github.com/anishsingh20/learned_optimization/tree/main">Github repo</a></li><li><a href="https://colab.research.google.com/github/google/learned_optimization/blob/main/learned_optimization/research/general_lopt/Demo_for_training_a_model_with_a_learned_optimizer.ipynb#scrollTo=RQBACAPQZyB-">Demo_for_training_a_model_with_a_learned_optimizer</a> python package.</li></ol><p><a href="https://github.com/anishsingh20/learned_optimization/tree/main">GitHub - anishsingh20/learned_optimization</a></p><blockquote>Save this post and try the example mentioned in the <a href="https://colab.research.google.com/github/google/learned_optimization/blob/main/learned_optimization/research/general_lopt/Demo_for_training_a_model_with_a_learned_optimizer.ipynb#scrollTo=RQBACAPQZyB-">Google Colab</a> notebook for various Deep learning tasks and projects. Do share the results and the findings in the comments here. Let’s build the future of AI together.</blockquote><p><strong>Share this post with your fellow AI, deep learning enthusiasts and experiment and play around with </strong><a href="https://arxiv.org/abs/2211.09760?utm_campaign=The%20Batch&amp;utm_source=hs_email&amp;utm_medium=email&amp;_hsenc=p2ANqtz-98Z0EEpyu-3WG5zBBBGen28QtQATna5cMllWRxO7eBqQ-IMiqGxR4sZnxp2nFzIAc70bQt"><strong>VeLO</strong></a><strong>.</strong></p><h3><strong>Connect with Me on Social Media:</strong></h3><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow to get Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><p>LinkedIn: [<a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">https://www.linkedin.com/in/anish-singh-walia-924529103/</a>]</p><p>GitHub: [<a href="https://github.com/anishsingh20">https://github.com/anishsingh20</a>]</p><p><a href="https://github.com/anishsingh20">anishsingh20 - Overview</a></p><p>Let’s build the future of AI together. Checkout my other blog posts related to developing AI applications, ChatGPT, OpenAI, and much more below:</p><p><a href="https://medium.com/@anishsinghwalia">Anish Singh Walia - Medium</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8c2c2d93da2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build Your Personal ChatGPT Bot with Streamlit and OpenAI APIs]]></title>
            <link>https://anishsinghwalia.medium.com/build-your-personal-chatgpt-bot-with-streamlit-and-openai-apis-84a05ab6929b?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/84a05ab6929b</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[ai-development]]></category>
            <category><![CDATA[streamlit]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Tue, 25 Jul 2023 04:25:04 GMT</pubDate>
            <atom:updated>2024-01-11T13:25:48.226Z</atom:updated>
            <content:encoded><![CDATA[<h3>Personal ChatGPT Bot with Streamlit: Build Your AI Assistant Web app</h3><blockquote>Do you want to have your very own AI-powered web-app that can understand your questions and provide helpful answers? Look no further! With the power of Python and <a href="https://docs.streamlit.io/">Streamlit</a> combined with OpenAI APIs, building a personalized ChatGPT bot is easier than you might think.</blockquote><p><a href="https://www.streamlit.io/">Streamlit</a> is an <strong>open-source</strong> <strong>Python library</strong> that makes it easy to create and share beautiful, custom <strong>web apps</strong> for machine learning and data science. You can build and deploy powerful data apps in just a few minutes. So let’s get started!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YpaBqKxvqZJwxlseivr4_w.png" /><figcaption>Streamlit platform | Source: streamlit.io</figcaption></figure><blockquote>To add more value and as a USP of my blog, I share this month’s bonus tip or best productivity tools that are cheap, effective, and a game changer, which I personally use and prefer. So do check them out and use them. Keep learning and growing.</blockquote><p>Here is the Bonus tip for you all:</p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><h3>Index</h3><blockquote><strong>Introduction — The Power of Personal AI Assistants</strong></blockquote><blockquote><strong>Setting Up OpenAI API Key and Streamlit</strong></blockquote><blockquote><strong>Building the ChatGPT Bot with Streamlit</strong></blockquote><blockquote><strong>Running the ChatGPT Bot using Streamlit</strong></blockquote><blockquote><strong>Model Parameters for Customizing Responses</strong></blockquote><blockquote><strong>Making the WebApp Visually Appealing</strong></blockquote><blockquote><strong>Frequently Asked Questions (FAQs)</strong></blockquote><blockquote><strong>Conclusion</strong></blockquote><blockquote>By the end of this post, you should be able to run a ChatGPTbot web app easily like the below:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bEDb7V2A0jJt6srDhNIw4A.png" /><figcaption>Sample screenshot from the app</figcaption></figure><h3>1. Introduction — The Power of Personal AI Assistants</h3><p>Imagine having your AI-powered chatbot web app ready to answer questions, generate creative content, and assist you in your daily tasks. Thanks to OpenAI’s powerful language models and easy-to-use <a href="https://beta.openai.com/docs/">APIs</a>, building a personal AI assistant is now within reach of any Python developer. This tutorial will show you how to create your ChatGPT bot using the <a href="https://www.streamlit.io/">Streamlit framework</a> and OpenAI’s <a href="https://platform.openai.com/docs/models/gpt-3-5">GPT-3.5-turbo</a> model.</p><h3>2. Setting Up OpenAI API and Streamlit</h3><p>Before building the ChatGPT bot, ensure access to OpenAI’s API. You can sign up for an <a href="https://platform.openai.com/account/api-keys">API key</a> on the <a href="https://platform.openai.com/account/api-keys">OpenAI website</a> if you haven’t already. Once you have your <a href="https://platform.openai.com/account/api-keys">API key</a>, set it up as an environment variable in your development environment on the terminal/shell:</p><pre>export OPENAI_API_KEY=&lt;YOUR_API_KEY&gt;</pre><p>Next, install the required libraries and set up your Python environment:</p><pre>pip install streamlit openai</pre><h3>3. Building the ChatGPT Bot with Streamlit</h3><p>Let’s dive into the code to build our chatbot using <a href="https://docs.streamlit.io/">Streamlit</a>. Create a Python file (e.g., chatbot_app.py) and follow along:</p><pre>import streamlit as st<br>import openai<br>import os<br><br># Set your OpenAI API key here<br>openai.api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)<br><br># Function to interact with the GPT-3.5-turbo model with tunable parameters<br>def generate_response(prompt, temperature=0.7, max_tokens=256, top_p=0.9, n=2, stop=None, frequency_penalty=0.9, presence_penalty=0.9, chat_history=None):<br>    if chat_history is None:<br>        chat_history = []<br><br>    messages = [<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt},<br>    ]<br>    messages.extend(chat_history)<br><br>    response = openai.ChatCompletion.create(<br>        model=&quot;gpt-3.5-turbo&quot;,<br>        messages=messages,<br>        temperature=temperature,<br>        max_tokens=max_tokens,<br>        top_p=top_p,<br>        n=n,<br>        stop=stop,<br>        frequency_penalty=frequency_penalty,<br>        presence_penalty=presence_penalty<br>    )<br><br>    return response[&#39;choices&#39;][0][&#39;message&#39;][&#39;content&#39;]<br><br># Streamlit app header and title<br> # tattooed geek logo<br>logo1 = &#39;https://miro.medium.com/v2/resize:fit:180/1*ypRBA86IBBbZbti76vm4Hg.png&#39;<br># Streamlit app header and title<br>st.set_page_config(page_title=&quot;Personal ChatGPT bot | By Anish Singh Walia&quot;, page_icon=logo1 , layout=&quot;wide&quot;)<br><br><br>st.write(&quot;# Personal Chatbot with GPT-3.5-turbo :sunglasses: &quot;)<br>st.write(&quot;Made with love by - [Anish Singh Walia](https://anishsinghwalia.medium.com/)&quot;)<br>st.write(&quot;Welcome to your personal chatbot app! Type your message below:&quot;)<br><br><br># Sidebar with social profiles and model parameters<br>st.sidebar.markdown(&quot;# Follow me on my Social Profiles&quot;)<br>st.sidebar.markdown(<br>    &quot;&quot;&quot;&lt;a href=&quot;https://github.com/anishsingh20&quot; target=&quot;_blank&quot;&gt;&lt;img src=&quot;https://cdn-icons-png.flaticon.com/512/25/25231.png&quot; alt=&quot;GitHub&quot; width=&quot;60px&quot;&gt;&lt;/a&gt;<br>    &lt;a href=&quot;https://www.linkedin.com/in/anish-singh-walia-924529103/&quot; target=&quot;_blank&quot;&gt;&lt;img src=&quot;https://cdn1.iconfinder.com/data/icons/logotypes/32/circle-linkedin-512.png&quot; alt=&quot;LinkedIn&quot; width=&quot;60px&quot;&gt;&lt;/a&gt;<br>    &lt;a href=&quot;https://medium.com/@anishsinghwalia&quot; target=&quot;_blank&quot;&gt;&lt;img src=&quot;https://cdn1.iconfinder.com/data/icons/social-media-circle-7/512/Circled_Medium_svg5-512.png&quot; alt=&quot;Medium&quot; width=&quot;60px&quot;&gt;&lt;/a&gt;<br>    &lt;a href=&quot;https://instagram.com/cali_br20&quot; target=&quot;_blank&quot;&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/e/e7/Instagram_logo_2016.svg/2048px-Instagram_logo_2016.svg.png&quot; alt=&quot;Instagram&quot; width=&quot;60px&quot;&gt;&lt;/a&gt;<br>    &quot;&quot;&quot;,<br>    unsafe_allow_html=True,<br>)<br><br># HTML sidebar to fine-tune model&#39;s parameters to customize the bot&#39;s responses.<br>st.sidebar.markdown(&quot;# Model Parameters&quot;)<br>temperature = st.sidebar.slider(&quot;Temperature&quot;, 0.0, 1.0, 0.7, 0.1)<br>max_tokens = st.sidebar.number_input(&quot;Max Tokens&quot;, 50, 500, 256, step=50)<br>top_p = st.sidebar.slider(&quot;Top P&quot;, 0.1, 1.0, 0.9, 0.1)<br>n = st.sidebar.number_input(&quot;N&quot;, 1, 5, 2, step=1)<br>stop = st.sidebar.text_input(&quot;Stop&quot;, &quot;&quot;)<br>frequency_penalty = st.sidebar.slider(&quot;Frequency Penalty&quot;, 0.0, 1.0, 0.9, 0.1)<br>presence_penalty = st.sidebar.slider(&quot;Presence Penalty&quot;, 0.0, 1.0, 0.9, 0.1)<br><br># Main app where user enters prompt and gets the response<br>user_input = st.text_area(&quot;You:&quot;, &quot;&quot;, key=&quot;user_input&quot;)<br>generate_button = st.button(&quot;Generate Response&quot;)<br><br><br># Chat history<br>messages = []<br>if user_input.strip() != &quot;&quot;:<br>    messages.append({&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: user_input})<br>    response = generate_response(user_input, temperature, max_tokens, top_p, n, stop, frequency_penalty, presence_penalty)<br>    messages.append({&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: response})<br><br>st.subheader(&quot;Chat History&quot;)<br>for message in messages:<br>    if message[&quot;role&quot;] == &quot;user&quot;:<br>        st.text_area(&quot;You:&quot;, value=message[&quot;content&quot;], height=50, max_chars=200, key=&quot;user_history&quot;, disabled=True)<br>    else:<br>        st.text_area(&quot;Jarvis:&quot;, value=message[&quot;content&quot;], height=500, key=&quot;chatbot_history&quot;)<br><br># Additional styling to make the app visually appealing<br>st.markdown(<br>    &quot;&quot;&quot;<br>    &lt;style&gt;<br>        body {<br>            font-family: Montserrat, sans-serif;<br>        }<br>        .stTextInput&gt;div&gt;div&gt;textarea {<br>            background-color: #f0f0f0;<br>            color: #000;<br>        }<br>        .stButton button {<br>            background-color: #4CAF50;<br>            color: white;<br>            font-weight: bold;<br>        }<br>        .stTextArea&gt;div&gt;textarea {<br>            resize: none;<br>        }<br>        .st-subheader {<br>            margin-top: 20px;<br>            font-size: 16px;<br>        }<br>        .stTextArea&gt;div&gt;div&gt;textarea {<br>            height: 100px;<br>        }<br>    &lt;/style&gt;<br>    &quot;&quot;&quot;,<br>    unsafe_allow_html=True,<br>)</pre><p>In this code, we set up our Streamlit app with a header, title, and text input area where users can interact with the chatbot. We also create a sidebar with social profiles and model parameters that we can fine-tune to customize the bot’s responses.</p><ul><li><strong>Streamlit App Setup</strong>: In this part, we set up the Streamlit app’s user interface. We create a simple text area where users can type their messages, and a button to generate the AI’s response. We also provide sliders and text input fields for tuning the model parameters.</li><li><strong>GPT-3.5-turbo Model Interaction</strong>: This part includes the function generate_response responsible for communicating with the GPT-3.5-turbo model through OpenAI&#39;s chatcompletion.create() API.</li></ul><blockquote>You can explore more about <strong>Streamlit’s</strong> documentation <a href="https://docs.streamlit.io/">here</a>.</blockquote><h3>4. Running the ChatGPT Bot using Streamlit</h3><p>To run the ChatGPT bot, execute the following command in your terminal or command prompt:</p><pre>streamlit run chatbot_app.py</pre><p>This will launch the Streamlit app in your default web browser. You can now start interacting with your personal ChatGPT bot!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nZZxbgW4BIPQuvHDceYFXQ.png" /><figcaption>The deployed WebApp</figcaption></figure><h3>5. Model Parameters for Customizing Responses</h3><p>The GPT-3.5-turbo model allows us to adjust several parameters to control the chatbot&#39;s behavior. Here’s a brief explanation of each parameter:</p><blockquote><strong>Temperature:</strong> Controls the randomness of the responses. Higher values (e.g., 1.0) make the output more diverse, while lower values (e.g., 0.2) make it more focused and deterministic.</blockquote><blockquote><strong>Max Tokens</strong>: Limits the length of the response generated by the model.</blockquote><blockquote><strong>Top P</strong>: Specifies the cumulative probability threshold for choosing the next token. Higher values (e.g., 0.9) result in more focused responses.</blockquote><blockquote><strong>N</strong>: Determines the number of different responses generated by the model, which helps in exploring different possibilities.</blockquote><blockquote><strong>Stop</strong>: Allows us to specify a stopping phrase to indicate the end of the response.</blockquote><blockquote><strong>Frequency Penalty</strong>: Controls the model’s likelihood of repeating the same response.</blockquote><blockquote><strong>Presence Penalty</strong>: Controls how much the model considers using a token that hasn’t been mentioned in the conversation.</blockquote><p>You can read more about fine-tuning model parameters here in my previous post on <a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></p><p><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></p><h3>6. Making the App Visually Appealing</h3><p>To make the app visually appealing, we have added some additional styling. The text area for the user’s input is set with a light gray background, and the chatbot’s responses are displayed in a larger text area. The button for generating the response is styled with a bold green color.</p><h3>7. Frequently Asked Questions (FAQs)</h3><h4>Q1: What is the GPT-3.5-turbo model?</h4><p>GPT-3.5-turbo is a highly advanced language model developed by OpenAI. It is designed to understand natural language and generate human-like text responses. With fewer tokens and lower costs than previous models, it provides an efficient and cost-effective solution for various AI applications.</p><h4>Q2: Can I deploy this ChatGPT bot on Shell/terminal?</h4><p>You can find a detailed guide on deploying your ChatGPT bot on shell/terminal in my previous article, <a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">Build and Deploy Your Personal Jarvis ChatGPT Bot in Python with ChatGPT API on MacOS</a>.</p><h4>Q3: Where can I learn more about model parameters in OpenAI API?</h4><p>For a comprehensive understanding of model parameters and how they influence the behavior of language models, check out our article on <a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a>.</p><h4>Q4: How can I ensure the security of my OpenAI API key?</h4><p>To ensure the security of your OpenAI API key, follow the best practices outlined in my article <a href="https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a">Safeguarding Your AI: Best Practices for Securing Your OpenAI API Key</a>.</p><h3>8. Conclusion</h3><p>Congratulations! You’ve successfully built your ChatGPT bot using the Streamlit framework and OpenAI’s GPT-3.5-turbo model. Now, you can interact with your AI assistant, customize its responses, and even deploy it on macOS or any other platform. Feel free to experiment with different model parameters to achieve the desired responses.</p><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow to get Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><p>Connect with me on <a href="https://github.com/anishsingh20">GitHub</a> and <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> to stay updated with more exciting AI content, collaborate on projects, and build together.</p><p><a href="https://github.com/anishsingh20">anishsingh20 - Overview</a></p><p><strong>Read the below blog post on how Generative AI + Streamlit is the best combination for AI developers and AI enthusiasts for developing and quickly deploying LLMs and complex generative AI applications over the web.</strong></p><p><a href="https://blog.streamlit.io/generative-ai-and-streamlit-a-perfect-match/">Generative AI and Streamlit: A perfect match</a></p><p><a href="https://github.com/anishsingh20/Streamlit-Webapp-ChatGPT/">Github Repo</a> for this project:</p><p><a href="https://github.com/anishsingh20/Streamlit-Webapp-ChatGPT/">GitHub - anishsingh20/Streamlit-Webapp-ChatGPT: Your own customisable ChatGPT Webapp made in Python using Streamlit</a></p><p>For more information on the OpenAI API and the GPT-3.5-turbo model, you can visit the <a href="https://beta.openai.com/docs/">OpenAI API documentation</a> and the <a href="https://openai.com/">OpenAI platform</a>.</p><blockquote>Happy coding, and have fun exploring the possibilities of conversational AI!</blockquote><p><strong>Links to previous articles:</strong></p><ul><li><a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">Build and Deploy Your Personal Jarvis ChatGPT Bot in Python with ChatGPT API on MacOS</a></li></ul><p><a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">Build and Deploy Your Personal Jarvis ChatGPT Bot in Python with ChatGPT API on MacOS</a></p><ul><li><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></li></ul><p><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></p><ul><li><a href="https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a">Safeguarding Your AI: Best Practices for Securing Your OpenAI API Key</a></li></ul><p><a href="https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a">Safeguarding Your AI: Best Practices for Securing Your OpenAI API Key</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84a05ab6929b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Safeguarding Your AI: Best Practices for Securing Your OpenAI API Key]]></title>
            <link>https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/67e5e585c59a</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[openai-api]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Sun, 23 Jul 2023 03:41:10 GMT</pubDate>
            <atom:updated>2024-01-11T13:31:49.065Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Unleashing the Power of AI with Peace of Mind</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HCAFhRkRkio_pu_7KozEMg.png" /><figcaption>Secure your API Keys | Ways to secure — Designed by <a href="https://medium.com/u/3bfe9db19781">Anish Singh Walia</a> in Canva</figcaption></figure><h3>Introduction</h3><p>As AI enthusiasts, researchers, engineers, and students, we are all captivated by the potential of Artificial Intelligence to transform our world. The <a href="https://platform.openai.com/docs/guides/gpt/completions-api">OpenAI API</a>, particularly <a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">ChatGPT,</a> empowers us to build exciting applications that leverage the prowess of language models. However, amidst this excitement, it is crucial to prioritize security and protect our API keys from potential misuse or unauthorized access.</p><p>This article will explore the best practices for safeguarding your OpenAI API key. We’ll delve into the importance of API key essential safety, common vulnerabilities, and practical steps to fortify your applications.</p><p>Whether you’re building a <a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">ChatGPT-powered bot</a> or any other AI-based application, these practices will help you preserve the integrity and confidentiality of your AI projects.</p><p>Here is the Bonus tip for you all:</p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /></figure><h3>Why API Key Safety Matters?</h3><p>Before diving into the best practices, let’s understand why API key safety is paramount. <strong><em>Your API key acts as a secret passphrase that grants access to your OpenAI account and the corresponding resources. If malicious actors gain access to your key, they can misuse it in various ways:</em></strong></p><ol><li><strong>Unauthorized Usage:</strong> Attackers can use your API key to make requests and incur costs on your account, exhausting your usage limit or even leading to financial losses.</li><li><strong>Data Breach:</strong> Compromised API keys might provide unauthorized access to sensitive data that your application processes, leading to potential data breaches and privacy violations.</li><li><strong>Misuse of Model:</strong> Malicious entities can exploit your API key to use ChatGPT for unethical or harmful activities, which could damage your organization’s reputation.</li></ol><p>Now that we understand the risks, let’s delve into the best practices to secure your <a href="https://platform.openai.com/account/api-keys">OpenAI API key</a>.</p><p><a href="https://platform.openai.com/account/api-keys">OpenAI Platform</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F81xwEHX23zhvy%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F81xwEHX23zhvy%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2F81xwEHX23zhvy%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="235" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c2f387777b41592077ce3bda803b0c2/href">https://medium.com/media/7c2f387777b41592077ce3bda803b0c2/href</a></iframe><h3><strong>1. Use Environment Variables: Shielding Your Secrets</strong></h3><p>Hardcoding your API key directly in your application’s source code is a risky practice. Instead, use <a href="https://www.geeksforgeeks.org/environment-variables-in-linux-unix/#:~:text=Environment%20variables%20or%20ENVs%20basically,and%20interactions%20with%20other%20processes."><strong>environment variables</strong></a> to store and access your API key securely. This prevents accidental exposure when sharing code repositories or deploying your application.</p><p>Hardcoding your API key directly in your application’s source code is a risky practice. Instead, use environment variables to store and access your API key securely. This prevents accidental exposure when sharing code repositories or deploying your application.</p><p>One popular way to manage environment variables in Python is using the python-dotenv package. This lightweight library allows us to load variables from a .env file into the application&#39;s environment.</p><p>Here’s how you can use python-dotenv to store your API key securely:</p><p><strong>Step 1</strong>: Install the python-dotenv package using pip:</p><pre>pip install python-dotenv</pre><p><strong>Step 2:</strong> Add your API key to a.env File in your project directory as follows:</p><pre>echo &quot;OPENAI_API_KEY=YOUR_OPENAI_API_KEY&quot; &gt;&gt; .env</pre><p>This command appends the OPENAI_API_KEY=YOUR_OPENAI_API_KEY line to the .env file, creating the file if it does not already exist. Make sure to replace YOUR_OPENAI_API_KEY with your actual OpenAI API key.</p><p>After running this command, you can open the .env file and verify that the OPENAI_API_KEY variable has been added with your API key as the value.</p><p><strong>Step 3</strong>: In your Python code, load the environment variables using dotenv:</p><pre>import os<br>from dotenv import load_dotenv  <br><br># Load environment variables from .env file<br>load_dotenv()<br># Access the API key using the variable name defined in the .env file<br>api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)</pre><p>By using python-dotenv, your <strong>API key</strong> is now safely stored in a separate file and can be accessed in your code through the os.getenv() method.</p><blockquote>Remember to add the .env file to your .gitignore file to avoid inadvertently exposing it when sharing your code with others on <a href="https://github.com/anishsingh20">Github</a>.</blockquote><p>Using environment variables in this way is essential in keeping your API key secure and your AI applications protected from potential threats.</p><h3><strong>2. Limit Access: Principle of Least Privilege</strong></h3><p>Follow the principle of least privilege by granting your API key only the necessary permissions.</p><p>OpenAI allows you to customize<strong> API key permissions</strong>, ensuring it can only perform specific actions, such as reading models or generating responses. <strong><em>Restricting permissions minimizes the potential impact of unauthorized access.</em></strong></p><p>Here’s an example of creating a new API key from an existing key with limited permissions using the OpenAI Python library:</p><pre>import openai<br>import os<br>from dotenv import load_dotenv  <br><br># Load environment variables from .env file<br>load_dotenv()<br><br># Access the existing API key using the variable name defined in the .env file<br>api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)<br><br><br># Create an API key with limited permissions - .create() and .read() permissions<br>key_info = openai.Key.create(<br>    scopes=[&quot;chat.completions.create&quot;, &quot;chat.models.read&quot;],<br>    name=&quot;Limited Key&quot;<br>)<br><br># Print the new API key<br>print(&quot;Limited API Key:&quot;, key_info[&quot;secret&quot;])</pre><p>Now you can follow the same procedure to store this new modified API key to an environment variable as explained in the above section.</p><h3><strong>3. Secure Transmission: Always Use HTTPS</strong></h3><p>Ensure that all communication with the <a href="https://platform.openai.com/docs/guides/gpt/completions-api">OpenAI API</a> occurs over a secure HTTPS connection. <strong><em>HTTPS encrypts data during transit, preventing eavesdropping and man-in-the-middle attacks.</em></strong></p><p>In Python, you can use the popular requests library to make secure API calls:</p><pre>import requests<br>import os<br>from dotenv import load_dotenv  <br><br># Load environment variables from .env file<br>load_dotenv()<br># Access the API key using the variable name defined in the .env file<br>api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)<br><br># defininig the HTTP headers for API reqyest<br>headers = {<br>    &quot;Authorization&quot;: f&quot;Bearer {api_key}&quot;<br>}<br><br>#sending a POST request using the v1 GPT /completions API with the requied headers and prompt from user<br>response = requests.post(<br>    &quot;https://api.openai.com/v1/engines/davinci-codex/completions&quot;,<br>    headers=headers,<br>    json={<br>        &quot;prompt&quot;: &quot;Write a Python function to calculate the factorial of a number.&quot;,<br>        &quot;max_tokens&quot;: 100<br>    }<br>)<br><br>print(response.json())</pre><ol><li>headers: It is a dictionary (key-value pairs) that will specify the HTTP headers for the API request. Headers provide additional information to the server handling the request.</li><li>&quot;Authorization&quot;:: This is the key in the headers dictionary. In this case, it indicates that we are setting the &quot;Authorization&quot; header.</li><li>f&quot;Bearer {api_key}&quot;: This part uses an f-string (formatted string literal) in Python to include the value of the api_key variable in the string. The f-string allows us to embed the value of api_key directly into the string.</li><li>api_key: This is a variable that contains your OpenAI API key. It is being used to replace the {api_key} placeholder in the f-string with the actual value of your API key.</li></ol><p>So, after this line of code, the headers dictionary will have an &quot;Authorization&quot; key with a value in the format &quot;Bearer YOUR_OPENAI_API_KEY&quot;, where &quot;YOUR_OPENAI_API_KEY&quot; will be replaced with the actual value of your OpenAI API key. <strong><em>This header is crucial for authenticating your API request and allowing access to the OpenAI API using your API key.</em></strong></p><p>Sample Output:</p><pre>{<br>    &quot;id&quot;: &quot;chatcmpl-6p9XYPYSTTRi0xEviKjjilqrWU2Ve&quot;,<br>    &quot;object&quot;: &quot;chat.completion&quot;,<br>    &quot;created&quot;: 1677649420,<br>    &quot;model&quot;: &quot;gpt-3.5-turbo&quot;,<br>    &quot;usage&quot;: {<br>        &quot;prompt_tokens&quot;: 26,<br>        &quot;completion_tokens&quot;: 31,<br>        &quot;total_tokens&quot;: 57<br>    },<br>    &quot;choices&quot;: [<br>        {<br>            &quot;message&quot;: {<br>                &quot;role&quot;: &quot;system&quot;,<br>                &quot;content&quot;: &quot;You are a bot that calculates factorials of numbers.&quot;<br>            },<br>            &quot;finish_reason&quot;: &quot;stop&quot;,<br>            &quot;index&quot;: 0<br>        },<br>        {<br>            &quot;message&quot;: {<br>                &quot;role&quot;: &quot;user&quot;,<br>                &quot;content&quot;: &quot;Write a Python function to calculate the factorial of a number.&quot;<br>            },<br>            &quot;finish_reason&quot;: &quot;stop&quot;,<br>            &quot;index&quot;: 1<br>        },<br>        {<br>            &quot;message&quot;: {<br>                &quot;role&quot;: &quot;assistant&quot;,<br>                &quot;content&quot;: &quot;Sure! Here&#39;s a Python function to calculate the factorial of a number:\n\n```python\n<br>def factorial(n):\n    if n == 0:\n        return 1\n    else:\n        return n * factorial(n-1)\n```\n```&quot;<br>            },<br>            &quot;finish_reason&quot;: &quot;stop&quot;,<br>            &quot;index&quot;: 1<br>        }<br>    ]<br>}</pre><h3><strong>4. Audit and Monitoring: Keep an Eye Out</strong></h3><p>Regularly audit and monitor the usage of your <a href="https://platform.openai.com/account/usage">API key</a>. OpenAI provides usage statistics and logs that can help you detect any suspicious activities early on. Keeping a close eye on your API key usage can prevent surprises.</p><p><a href="https://platform.openai.com/account/usage">OpenAI Platform</a></p><h3>FAQs:</h3><p><strong>Q1: Can I regenerate my API key?</strong></p><p>Yes, you can regenerate your API key if you suspect it has been compromised. The old key will be invalidated, and you can update your application with the new key.</p><p><strong>Q2: Is it safe to share my API key with my team?</strong></p><p>While sharing API keys with your trusted team members is generally safe, please exercise caution and limit access to those requiring it for development or testing purposes.</p><p><strong>Q3: Can I use the same API key for multiple applications?</strong></p><p>It’s best practice to use separate API keys for different applications, as it allows you to manage permissions more granularly and provides better control over usage.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F3o6MbmTd7iUDK5AiHu%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2F3o6MbmTd7iUDK5AiHu%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExb2pzM25wMmJjeTMyamlmaXNlN3Fqb21uNDdoMHllYXIzcThrcWhlMyZlcD12MV9naWZzX2dpZklkJmN0PWc%2F3o6MbmTd7iUDK5AiHu%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="328" frameborder="0" scrolling="no"><a href="https://medium.com/media/6bcf1bb41d56c415d29f17d4a10bb032/href">https://medium.com/media/6bcf1bb41d56c415d29f17d4a10bb032/href</a></iframe><h3>Conclusion</h3><p>By implementing these best practices, you can ensure the safety and security of your OpenAI API key. Securing your API key is the first step in building reliable and ethical AI applications.</p><p>As AI enthusiasts, researchers, and developers, let’s harness the potential of AI while safeguarding it from potential risks. Happy coding and building your AI-powered wonders!</p><blockquote>Remember, the power of AI comes with great responsibility — safeguard it with care.</blockquote><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow to get Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><p>I urge the readers to use the above methods to secure their API keys and build their <a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">ChatGPT bots.</a></p><p><a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">Build and Deploy Your Personal Jarvis ChatGPT Bot in Python with ChatGPT API on MacOS</a></p><p>Comment about how you secure your API keys and what different ways you use to secure them.</p><p>Connect with me on my social profiles like <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> &amp; <a href="https://github.com/anishsingh20">Github</a>. And let’s collaborate.</p><p><a href="https://github.com/anishsingh20">anishsingh20 - Overview</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=67e5e585c59a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Parameters in OpenAI API]]></title>
            <link>https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/161a5b1f8129</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[openai-api]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Wed, 19 Jul 2023 04:07:39 GMT</pubDate>
            <atom:updated>2024-01-11T13:27:16.910Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Let’s break down the ChatCompletion.create() Function and find the sweet-spot for the parameter values together.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FE8rtvw8ZDKrB6Ip6pgFlw.png" /><figcaption>High-Level overview of model Parameters and their Values | Designed in Canva by <a href="https://medium.com/u/3bfe9db19781">Anish Singh Walia</a></figcaption></figure><h3>Introduction</h3><p>In this blog post, we will delve into the inner workings of the openai.ChatCompletion.create() function in the <a href="https://platform.openai.com/docs/api-reference">OpenAI API</a>.</p><p>Whether you’re a geek, student, engineer, or technologist, this guide will provide you with the knowledge to harness the capabilities of the ChatGPT API. You will also find a link to another post towards the end using which you can build your very own <a href="https://medium.com/@anishsinghwalia/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">ChatGPT personal assistant on terminal/shell</a></p><p>This function plays a crucial role in generating most coherent responses from the ChatGPT model.</p><blockquote>This blog will also give you the MAGIC numbers or the MAGIC range for setting those parameters to get the best out of you ChatGPT application.</blockquote><blockquote><strong>NOTE:</strong> To add more value and as a USP of my blog, at the end of this blog post, I have designed and attached a cheat sheet/carousel of the model discussed here for you to use and share.</blockquote><blockquote>I’d love to share the high-quality PDF version of my blog’s cheatsheets with you — for free! Since Medium doesn’t allow uploading PDFs, I post them daily on my <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> page. Let’s connect and stay updated @ <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">https://www.linkedin.com/in/anish-singh-walia-924529103/</a></blockquote><blockquote>Also, I’ll share this month’s bonus tip or best productivity tools that are cheap, effective, and a game changer, which I personally use, prefer, and insist you all try. So do check them out and use them.</blockquote><p><strong>Here is the Bonus tip for you all. Let’s take a pause here before diving into the blog post.</strong></p><p><strong>Sharing knowledge is the key to growth and success. I want to shed some light on the productivity tools that have transformed my daily life and that I use on a day-to-day basis. These tools are my absolute favorite, are a great value for money and have helped me achieve my goals. Do open and check out these tools.</strong></p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><p>By understanding the various parameters it accepts and fine-tuning them, you can control the behavior and quality of the model&#39;s responses.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FCjmvTCZf2U3p09Cn0h%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FCjmvTCZf2U3p09Cn0h%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FCjmvTCZf2U3p09Cn0h%2Fgiphy.gif&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="409" frameborder="0" scrolling="no"><a href="https://medium.com/media/299d92410c85baea4abbb6ad6d7f7285/href">https://medium.com/media/299d92410c85baea4abbb6ad6d7f7285/href</a></iframe><h3>Table of Contents:</h3><ol><li>Understanding the <strong>openai.ChatCompletion.create()</strong> Function</li><li><strong>Temperature</strong>: Controlling andomness and <strong>diversity </strong>of the Responses</li><li><strong>Max Tokens</strong>: Limiting the Response Length</li><li><strong>Top P (Nucleus Sampling)</strong>: Controlling Response Quality</li><li><strong>n</strong>: Generating Multiple Responses</li><li><strong>Stop</strong>: Customizing Stop Conditions</li><li><strong>Frequency Penalty</strong>: Controlling Repetitive Responses</li><li><strong>Presence Penalty</strong>: Controlling Avoidance of Certain Topics</li><li><strong>Implementing Parameters in Python Code</strong></li><li><strong>Conclusion</strong></li></ol><h3>1. Understanding the openai.ChatCompletion.create() Function</h3><p>This function allows us to generate responses from the ChatGPT model by providing a series of messages as input.</p><p>In the <strong>openai.ChatCompletion.create()</strong> the function, the strings <strong>&quot;system&quot;</strong>, <strong>&quot;user&quot;</strong>, and <strong>&quot;assistant&quot;</strong> are used to define the role of each message within the conversation. These roles help provide context and structure to the model&#39;s understanding of the conversation flow.</p><p>The function call looks like this:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Who won the world series in 2020?&quot;},<br>        {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;The Los Angeles Dodgers won the World Series in 2020.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Where was it played?&quot;}<br>    ]<br>ptyth)</pre><p>Here, we pass the model parameter specifying the model (e.g., &quot;<strong>get-3.5-turbo</strong>&quot;). The messages parameter is an array of message objects, each having a &quot;<strong>role</strong>&quot; (&quot;<strong>system&quot;, &quot;user&quot;, or &quot;assistant</strong>&quot;) and &quot;<strong>content</strong>&quot; (the actual text of the message).</p><ol><li>“system”: The “system” role typically provides high-level instructions or context-setting messages. It helps guide the behavior of the assistant throughout the conversation. For example, a system message could be “You are a helpful assistant.”</li><li>“user”: The “user” role represents the messages or queries from the user or end-user interacting with the model. These messages contain the user’s input or questions. For instance, a user message could be “Tell me a fun fact.”</li><li>“assistant”: The “assistant” role represents the responses generated by the ChatGPT model. It includes the model’s generated replies or information provided to the user based on their queries. For example, an assistant message could be, “Sure, here is an interesting fact:”</li></ol><p>By assigning specific roles to each message, you establish a conversational context and guide the model’s understanding of the user’s input and the assistant’s responses. This allows for a more interactive and coherent conversation between the user and the AI model.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFpDHZWLT4%2Fview%3Fembed%26meta&amp;display_name=Canva&amp;url=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFpDHZWLT4%2Fview&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=canva" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/36074a47a788fc41a467a7683b8dd379/href">https://medium.com/media/36074a47a788fc41a467a7683b8dd379/href</a></iframe><h3>2. Temperature: Adding Randomness to the Responses</h3><p>The temperature parameter influences the randomness of the generated responses. A higher value, such as 0.8, makes the answers more <strong>diverse</strong>, while a lower value, like 0.2, makes them more <strong>focused</strong> and <strong>deterministic</strong>.</p><blockquote>A value between 0.2 and 0.8 can be effective. Lower values (e.g., 0.2) produce more focused and deterministic responses, while higher values (e.g., 0.8) allow for more randomness.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me a joke.&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Why don&#39;t scientists trust atoms?&quot;},<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;I don&#39;t know, why?&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Because they make up everything!&quot;}<br>  ],<br>  temperature=0.8<br>)</pre><p>Output:</p><pre>{<br>  ...<br>  &quot;choices&quot;: [<br>    {<br>      &quot;message&quot;: {<br>        &quot;content&quot;: &quot;Why did the chicken cross the road? To get to the other side!&quot;,<br>        &quot;role&quot;: &quot;assistant&quot;<br>      }<br>    }<br>  ],<br>  ...<br>}</pre><h3>3. Max Tokens: Limiting the Response Length</h3><p>The max_tokens parameter allows you to limit the length of the generated response. Setting an appropriate value allows you to control the response length and ensure it fits the desired context.</p><p><strong>Tokens</strong>: In NLP, a token refers to a unit of text the model reads. Tokens can be as short as one character or as long as one word, depending on the language and the specific model used. Tokens are common sequences of characters found in the text.</p><blockquote>Rule of thumb: 1 token is approximately 4 characters or 0.75 words for English text.</blockquote><blockquote>GPT models excel at understanding token relationships and generating the next token in a sequence.</blockquote><p>The model processes text by reading and generating tokens, and the number of tokens in an API call affects the cost and response time. Hence we need to set the max_tokens parameter and put a limit on the response length.</p><blockquote>It’s essential to remember that tokens are not equivalent to words, as they can also represent punctuation marks, spaces, or special characters.</blockquote><p>Use the below <a href="https://platform.openai.com/tokenizer">Tokenizer tool</a> to understand better how user text is broken down into tokens by the GPT model.</p><p><a href="https://platform.openai.com/tokenizer">OpenAI Platform</a></p><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Translate the following English text to French: &#39;Hello, how are you?&#39;&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure, here is the translation: &#39;Bonjour, comment ça va ?&#39;&quot;}<br>  ],<br>  max_tokens=20<br>)</pre><p>Output:</p><pre>{<br>  ...<br>  &quot;choices&quot;: [<br>    {<br>      &quot;message&quot;: {<br>        &quot;content&quot;: &quot;Bonjour, comment ça&quot;,<br>        &quot;role&quot;: &quot;assistant&quot;<br>      }<br>    }<br>  ],<br>  ...<br>}</pre><h3>4. Top P (Nucleus Sampling): Controlling Response Quality</h3><p>The top_p parameter, also known as nucleus sampling or &quot;penalty&quot; in the API, controls the diversity and quality of the responses. It limits the cumulative probability of the most likely tokens. Higher values like <strong>0.9</strong> allow more tokens, leading to <strong>diverse </strong>responses, while lower values like <strong>0.2</strong> provide more <strong>focused</strong> and <strong>constrained</strong> answers.</p><blockquote>A value between 0.3 and 0.9 is recommended. Higher values (e.g., 0.9) make the model consider a broader range of possibilities, while lower values (e.g., 0.3) make it more selective.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Who was the first man on the moon?&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;The first man on the moon was Neil Armstrong.&quot;},<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me more about him.&quot;}<br>  ],<br>  top_p=0.5<br>)</pre><p>Output:</p><pre>{<br>  ...<br>  &quot;choices&quot;: [<br>    {<br>      &quot;message&quot;: {<br>        &quot;content&quot;: &quot;Neil Armstrong was an astronaut who became famous for being the first person to walk on the moon. He was a part of the Apollo 11 mission in 1969.&quot;,<br>        &quot;role&quot;: &quot;assistant&quot;<br>      }<br>    }<br>  ],<br>  ...<br>}</pre><h3>5. n: Generating Multiple Responses</h3><p>The n parameter allows you to generate multiple alternative completions for a given conversation. By increasing the value of n, you can explore different response variations.</p><blockquote>Experiment with different values (e.g., 1, 2, or 3) to specify the number of alternative completions to generate. Increasing this value can offer more diverse response options.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Who is the best soccer player of all time?&quot;}<br>    ],<br>  n=3<br>)</pre><p>Output:</p><pre>{<br>  &#39;choices&#39;: [<br>    {<br>      &#39;message&#39;: {<br>        &#39;role&#39;: &#39;assistant&#39;,<br>        &#39;content&#39;: &#39;There are many great soccer players, but some of the most frequently mentioned ones include Pelé, Diego Maradona, and Lionel Messi.&#39;},<br>      &#39;finish_reason&#39;: &#39;stop&#39;,<br>      &#39;index&#39;: 0<br>    },<br>    {<br>      &#39;message&#39;: {<br>        &#39;role&#39;: &#39;assistant&#39;,<br>        &#39;content&#39;: &#39;When it comes to soccer, opinions may vary, but some popular choices for the best player of all time include Pelé, Diego Maradona, and Cristiano Ronaldo.&#39;},<br>      &#39;finish_reason&#39;: &#39;stop&#39;,<br>      &#39;index&#39;: 1<br>    },<br>    {<br>      &#39;message&#39;: {<br>        &#39;role&#39;: &#39;assistant&#39;,<br>        &#39;content&#39;: &#39;The best soccer player of all time is subjective and can vary depending on personal preferences. However, some widely recognized legends in the sport include Pelé, Diego Maradona, and Lionel Messi.&#39;},<br>      &#39;finish_reason&#39;: &#39;stop&#39;,<br>      &#39;index&#39;: 2<br>    }<br>  ]<br>}</pre><h3>6. Stop: Customizing stop Conditions</h3><p>The stop parameter allows you to specify a custom condition for the completion. You can provide a string or a list of strings that, when encountered in the generated response, will cause the model to stop generating further tokens. Here&#39;s an example</p><blockquote>Providing a list of stop words can help prevent the model from generating responses containing those specific words. Include relevant stop words based on your application.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>        {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are a helpful assistant.&quot;},<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Translate the following English text to French: &#39;Hello, how are you?&#39;&quot;}<br>    ],<br>  stop=&quot;Translate:&quot;<br>)</pre><p>Output:</p><pre>{<br>  &#39;id&#39;: &#39;chatcmpl-6p9XYPYSTTRi0xEviKjjilqrWU5Ve&#39;,<br>  &#39;object&#39;: &#39;chat.completion&#39;,<br>  &#39;created&#39;: 1677649420,<br>  &#39;model&#39;: &#39;gpt-3.5-turbo&#39;,<br>  &#39;usage&#39;: {&#39;prompt_tokens&#39;: 56, &#39;completion_tokens&#39;: 62, &#39;total_tokens&#39;: 118},<br>  &#39;choices&#39;: [<br>    {<br>      &#39;message&#39;: {<br>        &#39;role&#39;: &#39;assistant&#39;,<br>        &#39;content&#39;: &#39;Salut, comment ça va ?&#39;},<br>      &#39;finish_reason&#39;: &#39;stop&#39;,<br>      &#39;index&#39;: 0<br>    }<br>  ]<br>}</pre><h3>7. Frequency Penalty: Controlling Repetitive Responses</h3><p>The frequency_penalty parameter allows you to control the model&#39;s tendency to generate repetitive responses. Higher values, like <strong>1.0</strong>, <strong>encourage the model to explore more diverse and novel responses</strong>, while lower values, such as <strong>0.2, </strong>make the<strong> model more likely to repeat information.</strong></p><blockquote>Increasing this value (e.g., 0.6) encourages the model to avoid repeating the same words/phrases and can lead to more varied responses.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What are some tourist attractions in Paris?&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Some popular attractions in Paris are the Eiffel Tower, Louvre Museum, and Notre-Dame Cathedral.&quot;},<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me more about the Louvre Museum.&quot;}<br>  ],<br>  frequency_penalty=0.6<br>)</pre><p>Output:</p><pre>{<br>  ...<br>  &quot;choices&quot;: [<br>    {<br>      &quot;message&quot;: {<br>        &quot;content&quot;: &quot;The Louvre Museum is one of the world&#39;s largest and most visited museums. It is located in the heart of Paris and houses a vast collection of art and historical artifacts.&quot;,<br>        &quot;role&quot;: &quot;assistant&quot;<br>      }<br>    }<br>  ],<br>  ...<br>}</pre><h3>8. Presence Penalty: Controlling Avoidance of Certain Topics</h3><p>The presence_penalty parameter allows you to influence the model&#39;s avoidance of specific topics in its responses. <strong>Higher values, such as 1.0, make the model more likely to avoid mentioning particular </strong>topics provided in the user messages, while lower values, like <strong>0.2</strong>, make the <strong>model less concerned about preventing those topics</strong>.</p><blockquote>Increasing this value (<strong>e.g., 0.6</strong>) encourages the model to include more relevant details from the provided context and can enhance the specificity of responses.</blockquote><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me about your favorite book.&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;I enjoy reading science fiction novels.&quot;},<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What are your thoughts on &#39;Dune&#39; by Frank Herbert?&quot;}<br>  ],<br>  presence_penalty=0.8<br>)</pre><p>Output:</p><pre>{<br>  ...<br>  &quot;choices&quot;: [<br>    {<br>      &quot;message&quot;: {<br>        &quot;content&quot;: &quot;I&#39;m sorry, but I am not familiar with &#39;Dune&#39; by Frank Herbert.&quot;,<br>        &quot;role&quot;: &quot;assistant&quot;<br>      }<br>    }<br>  ],<br>  ...<br>}</pre><h3>10. Implementing Parameters in Python Code</h3><p>In this section, we provide a Python code snippet that combines the aforementioned parameters, allowing you to experiment and observe their effects on the generated responses.</p><p>Code Sample:</p><pre>response = openai.ChatCompletion.create(<br>  model=&quot;gpt-3.5-turbo&quot;,<br>  messages=[<br>      {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Tell me a fun fact.&quot;},<br>      {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: &quot;Sure, here is an interesting fact:&quot;}<br>  ],<br>  temperature=0.6,<br>  max_tokens=30,<br>  top_p=0.8,<br>  frequency_penalty=0.6,<br>  presence_penalty=0.8<br>)</pre><p>The expected output would be the response generated by the chatbot, which would be a completion of the conversation based on the provided context and the behavior of the model with the given parameters.</p><blockquote>Note: The code examples provided above assume the OpenAI Python library is installed and the appropriate API credentials are set up.</blockquote><blockquote>To know how to set up the Python environment and export the OpenAI API credentials you can refer my recent blog <a href="https://medium.com/@anishsinghwalia/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">here</a>.</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FIKc7Ld3LfO3haT3Gtv%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fufc-mma-tuf-team-mcgregor-vs-chandler-IKc7Ld3LfO3haT3Gtv&amp;image=https%3A%2F%2Fmedia3.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExdXVhcXU3ZGRvMHh5N3ZlazJuamc5N2wxcjhhZ3R5eTAwbm5rb3l6byZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FIKc7Ld3LfO3haT3Gtv%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="241" frameborder="0" scrolling="no"><a href="https://medium.com/media/53707a0a4703fba3b64bac54695bf6b2/href">https://medium.com/media/53707a0a4703fba3b64bac54695bf6b2/href</a></iframe><p>However, without actually running the code with a valid <a href="https://platform.openai.com/account/api-keys"><strong>OpenAI</strong> <strong>API key</strong></a>, you cannot build a ChatGPT application.</p><p>You should use and refer my recent blogpost on <a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7"><strong>building and deploying your Personal ChatGPT Bot in Python with ChatGPT API on MacOS</strong></a><strong> </strong>to create a simple personalterminal/shellchatGPT assistant bot to get started with exploring the capabilities of ChatGPT and innovate.</p><p>Once you have made and deployed the <a href="http://building and deploying your Personal ChatGPT Bot in Python with ChatGPT API on MacOS">terminal/shell</a> chatbot using the above post, on you machine, refer this blog to fine-tune and experiment with the chatGPT bot and the model’s parameters to see some varying, interesting and fun responses.</p><p><a href="https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7">Build and Deploy Your Personal ChatGPT Bot in Python with ChatGPT API on MacOS</a></p><h3>8. Conclusion</h3><p>In this blog post, we have explored the essential parameters of the <a href="https://platform.openai.com/docs/guides/gpt/chat-completions-api">openai.ChatCompletion.create()</a> function in the <a href="https://platform.openai.com/docs/guides/gpt/chat-completions-api">OpenAI API</a>.</p><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow for Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><p>By understanding and fine-tuning these parameters, you can shape the behavior and quality of the ChatGPT model&#39;s responses.</p><p>We covered the temperature, max_tokens, and top_p parameters, providing code samples and their respective outputs. Armed with this knowledge, we can now unlock the full potential of the OpenAI API and create more engaging and interactive chatbots.</p><p>Remember to experiment and iterate with different parameter values to achieve the desired results. <strong>Happy AI</strong>!</p><p>Connect with me on my social profiles like <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> &amp; <a href="https://github.com/anishsingh20">Github</a>. And let’s collaborate.</p><p><a href="https://github.com/anishsingh20">anishsingh20 - Overview</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=161a5b1f8129" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build and Deploy Your Personal Jarvis ChatGPT Bot in Python with ChatGPT API on MacOS]]></title>
            <link>https://anishsinghwalia.medium.com/build-and-deploy-your-personal-chatgpt-bot-in-python-with-chatgpt-api-on-macos-951a16aaaff7?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/951a16aaaff7</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatbots]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Sun, 16 Jul 2023 13:55:39 GMT</pubDate>
            <atom:updated>2024-01-11T13:28:25.376Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*omREvG5abgmk-0XCtF1nJA.png" /><figcaption>The Model’s high-level overview | Designed in Canva by <a href="https://medium.com/u/3bfe9db19781">Anish Singh Walia</a></figcaption></figure><blockquote>Supercharge Your Productivity with an AI Terminal Assistant!</blockquote><blockquote>If you love shell scripting and spend most of your time on <em>terminal/shell</em> like me, this post is definitely for you. You will have your own personal Shell Assistant that can give you answer to anything and make you more productive</blockquote><h3>INTRODUCTION</h3><p>Are you ready to harness the limitless potential of ChatGPT? Imagine having an AI-powered terminal/Shell assistant that provides instant support, automates complex tasks and saves valuable time. Need a tricky awk or Linux/shell command? Your assistant has got you covered!</p><p>With the <strong>gpt-3.5-turbo model</strong>, your Personal ChatGPT bot not only trains on the data you provide but also leverages the vast amount of knowledge from the extensive data it has access to. The possibilities are endless!</p><blockquote><strong>NOTE:</strong> To add more value and as a USP of my blog, at the end of this blog post, I have designed and attached a cheat sheet/carousel of the model discussed here for you to use and share.</blockquote><blockquote>I’d love to share the high-quality PDF version of my blog’s cheatsheets with you — for free! Since Medium doesn’t allow uploading PDFs, I post them daily on my <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> page. Let’s connect and stay updated @ <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">https://www.linkedin.com/in/anish-singh-walia-924529103/</a></blockquote><blockquote>Also, I’ll share this month’s bonus tip or best productivity tools that are cheap, effective, and a game changer, which I personally use, prefer, and insist you all try. So do check them out and use them.</blockquote><p>Here is the Bonus tip for you all:</p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><p>In just a few lines of Python code, you can have an assistant who answers any question, provides solutions, and gives personalized advice from your terminal or shell. It’s like having your very own AI sidekick!</p><p>Boost your productivity, streamline your workflows, and elevate your AI experience. So follow along and keep reading till the end.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FJmJMzlXOiI0dq%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FJmJMzlXOiI0dq%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FJmJMzlXOiI0dq%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="215" frameborder="0" scrolling="no"><a href="https://medium.com/media/6b879b30d187dac43aaa613ebda6b3fc/href">https://medium.com/media/6b879b30d187dac43aaa613ebda6b3fc/href</a></iframe><blockquote><a href="https://platform.openai.com/">OpenAI’s</a> ChatGPT API, fueled by the cutting-edge gpt-3.5-turbo model opens doors to this transformative technology. In this tutorial, I’ll guide you through the exhilarating process of building and deploying <strong>your Personal ChatGPT bot on your Macbook/Linux, which trains on the type of data you provide, along with the power of the immense amount of data that </strong>gpt-3.5-turbo<strong>uses. In just a few lines of python code. It&#39;s like having your assistant.</strong></blockquote><blockquote><strong>By the end of this post, you should be able to ask questions like below which could make your life a lot more productive and effective if you are someone who spends most of your day on </strong><strong>Shell/terminal. So follow along till the end.</strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1wyAvkYgSr2LHAEcSLJHYA.png" /><figcaption>Your personal bash assistant to make you more productive</figcaption></figure><h3>Table of Contents:</h3><ol><li><strong>Pre-requisites</strong></li><li><strong>Setting up the Environment</strong></li><li><strong>Obtaining OpenAI API access</strong></li><li><strong>Create a </strong><strong>/data Directory in the project’s current working directory</strong></li><li><strong>Building the ChatGPT Bot</strong></li><li><strong>Deploying the ChaGPT bot</strong></li><li><strong>Customizing and Enhancing Your ChatGPT Bot</strong></li><li><strong>Conclusion</strong></li></ol><h3>Model’s architecture and a high-level overview:</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFo9gl1iZM%2Fview%3Fembed%26meta&amp;display_name=Canva&amp;url=https%3A%2F%2Fwww.canva.com%2Fdesign%2FDAFo9gl1iZM%2Fview&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=canva" width="1920" height="1080" frameborder="0" scrolling="no"><a href="https://medium.com/media/191bf19c873ce408b49f97f7eac8617b/href">https://medium.com/media/191bf19c873ce408b49f97f7eac8617b/href</a></iframe><p>Let’s dive in!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FCC5MVO9Jx4RqMQRfvT%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FCC5MVO9Jx4RqMQRfvT%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FCC5MVO9Jx4RqMQRfvT%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/2350eb893f387de31d92a799f699339c/href">https://medium.com/media/2350eb893f387de31d92a799f699339c/href</a></iframe><h3>PRE-REQUISITES</h3><p>Before diving into the implementation, ensure you have the following:</p><ul><li>A Mac/Linux system(this tutorial is specifically tailored for <strong>macOS users)</strong></li><li><strong>Python 3.7 or higher installed</strong></li><li>Basic knowledge of Python programming</li><li>An <a href="https://platform.openai.com/"><strong>OpenAI</strong></a><strong> </strong>account with access to the <a href="https://platform.openai.com/account/api-keys"><strong>ChatGPT API</strong></a></li></ul><h3>Step 1: Setting Up the environment</h3><p>We need to set up a Python environment and install the necessary libraries. Open a terminal and create a new directory for your project.</p><pre>$ mkdir chatgpt-bot<br>$ cd chatgpt-bot<br>$ </pre><p>Here, we will use the three crucial libraries: <a href="https://platform.openai.com/"><strong>OpenAI</strong></a>,textract<strong> and </strong>glob<strong>to implement this.</strong></p><p><a href="https://platform.openai.com/"><strong>OpenAI</strong></a> is a leading artificial intelligence research organization that has developed the ChatGPT API, which allows us to interact with the powerful ChatGPT model. With the OpenAI API, we can send prompts and receive responses from the ChatGPT model, enabling us to create conversational chatbots.</p><p>You can learn more about <a href="https://platform.openai.com/"><strong>OpenAI</strong></a><strong> and its offerings </strong><a href="https://openai.com/"><strong>here</strong></a><strong>.</strong></p><p>The second textract Python library package provides text extraction capabilities from various file formats. It supports a wide range of file formats, including but not limited to:</p><ol><li>Text-based formats: TXT, CSV, JSON, XML, HTML, Markdown, and LaTeX.</li><li>Document formats: DOC, DOCX, XLS, XLSX, PPT, PPTX, ODT, and ODS.</li><li>eBook formats: EPUB, MOBI, AZW, and FB2.</li><li>Image formats with embedded text: JPG, PNG, BMP, GIF, TIFF, and PDF (both searchable and scanned).</li><li>Programming source code files: Python, C, C++, Java, JavaScript, PHP, Ruby, and more.</li></ol><p>The glob package in Python is a built-in module that provides a convenient way to search for files and directories using pattern matching. It allows you to find files matching a specified pattern, such as all files with a particular extension or specific naming patterns.</p><p>Next, let’s install the required Python libraries:</p><pre>$ pip install openai glob textract</pre><h3>Step 2: Obtaining OpenAI API Access</h3><p>To use the ChatGPT API, you’ll need an <a href="https://platform.openai.com/account/api-keys">OpenAI API key</a>. If you don’t have one, sign in to your <a href="https://platform.openai.com/account/api-keys">OpenAI account</a> and generate an API key from the dashboard.</p><p><a href="https://platform.openai.com/account/api-keys">OpenAI Platform</a></p><p>Once you have the API key, save it <strong>securely</strong> as an environment variable in your terminal. An environment variable is a variable that is set on your operating system, rather than within your application.</p><p>Securing your <a href="https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a">API keys</a> is paramount, and the post below explains various ways to secure your OpenAI API keys.</p><p><a href="https://anishsinghwalia.medium.com/safeguarding-your-ai-best-practices-for-securing-your-openai-api-key-67e5e585c59a">Safeguarding Your AI: Best Practices for Securing Your OpenAI API Key</a></p><p>To make the above OPENAI_API_KEY Persistent in your Linux/macOS machine’s environment variable use:</p><p>1. Run the following command in your terminal, replacing your key with your <a href="https://beta.openai.com/account/api-keys">API key</a>.</p><pre>echo &quot;export OPENAI_API_KEY=&#39;yourkey&#39;&quot; &gt;&gt; ~/.zshrc</pre><p>2. Update the shell with the new variable:</p><pre>source ~/.zshrc</pre><p>3. Confirm that you have set your environment variable using the following command from the terminal.</p><pre>echo $OPENAI_API_KEY</pre><h3>Step 3: Create a <strong>/data</strong> Directory in the project&#39;s current working directory</h3><p>Add all personal files there, containing anything from .txt to any .csv, or bot.docx files. The model will use the data inside this directory to answer your personal questions based on the data you store here. For example, I have created a<strong> .txt </strong>file inside this directory and added my <strong>resume</strong>:</p><figure><img alt="The /data directory which contains your data" src="https://cdn-images-1.medium.com/max/500/1*nNOCQpD6CL8-34IpI36nng.png" /><figcaption>The /data directory, which contains your data</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e-E1ULHdDPAVZkxRnCxKGg.png" /><figcaption>My data inside the My_data.txt file</figcaption></figure><h3>Step 4: Building the ChatGPT Bot</h3><p>Now, let’s write the code for our ChatGPT bot. Create a new Python file, such as <strong>chatGPTbot.py</strong>, in the current working directory of your Project, and write the following code logic. I urge the readers to understand the code and follow along, and if you have any doubts add in the comments section.</p><pre>import os<br>import glob<br>import openai<br>import textract<br><br>class Chatbot:<br>    def __init__(self):<br>        self.openai_api_key = os.getenv(&quot;OPENAI_API_KEY&quot;)<br>        self.chat_history = [] # chat_history list to keep the chat in memory of the model<br><br>    def append_to_chat_history(self, message): #appends the user&#39;s message to the chat history stored in the chat_history list.<br>        self.chat_history.append(message)<br><br>    def read_personal_file(self, file_path):<br>        try:<br>            text = textract.process(file_path).decode(&quot;utf-8&quot;) # convert the content inside files to plain text <br>            return text<br>        except Exception as e:<br>            print(f&quot;Error reading file {file_path}: {e}&quot;)<br>            return &quot;&quot;<br><br>    def collect_user_data(self): # collect local personal data to feed the model<br>        data_directory = &quot;./data&quot;<br>        data_files = glob.glob(os.path.join(data_directory, &quot;*.*&quot;)) # The function &quot;glob.glob(os.path.join(data_directory, &quot;*.*&quot;))&quot; <br>        # is used to retrieve a list of file paths that match a specified pattern within a given directory. <br>        # In this case *.*&quot;, which matches all files with any extension.<br><br>        user_data = &quot;&quot;<br>        for file in data_files:<br>            file_extension = os.path.splitext(file)[1].lower()<br>            if file_extension in (&quot;.pdf&quot;, &quot;.docx&quot;, &quot;.xlsx&quot;, &quot;.xls&quot;): #checks for the file extension<br>                user_data += self.read_personal_file(file) # convert the content of files to plain text and append<br>            else:<br>                with open(file, &quot;r&quot;, encoding=&quot;utf-8&quot;) as f: # the &quot;with&quot; statement is used to simplify exception handling and resource management when working with files<br>                    user_data += f.read() + &quot;\n&quot;      <br>                                               # the &quot;open&quot; function is called with the file name and mode (&#39;r&#39; for read). The returned file object is then assigned to the variable `user_data`. <br>                                                # The code inside the &quot;with&quot; block reads the contents of the file and prints it<br>        return user_data<br><br>    def create_chat_response(self, message):<br>        self.append_to_chat_history(message) # appends the user&#39;s message to the chat history stored in the chat_history list.<br><br>        user_data = self.collect_user_data()<br>        messages = [<br>            {&quot;role&quot;: &quot;system&quot;, &quot;content&quot;: &quot;You are the most helpful assistant.&quot;}, # provides high-level instructions or context-setting messages<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: message}, # user” role represents the messages or queries from the user<br>            {&quot;role&quot;: &quot;assistant&quot;, &quot;content&quot;: message}, # “assistant” role represents the responses generated by the ChatGPT model<br>        ]<br><br>        if user_data:<br>            messages.append({&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: user_data})<br><br>        # the main function that runs the ChatGPT model on the given parameters<br>        response = openai.ChatCompletion.create(<br>            model=&quot;gpt-3.5-turbo&quot;,<br>            messages=messages,<br>            temperature=0.7,<br>            max_tokens=256,<br>            top_p=0.9,<br>            n=2,<br>            stop=None,<br>            frequency_penalty=0.9,<br>            presence_penalty=0.9<br>        )<br><br>        self.append_to_chat_history(response.choices[0].message.content.strip()) <br>        return response.choices[0].message.content.strip() <br>        # add responses by model to model&#39;s memory/chat history to make the bot more interactive and intelligent<br><br>    def start_chatting(self):<br>        while True:<br>            user_input = input(&quot;User: &quot;)<br>            if user_input.lower() == &quot;exit&quot;:<br>                print(&quot;Jarvis: Goodbye boss, have a wonderfull day ahead!&quot;)<br>                break<br>            bot_response = self.create_chat_response(user_input)<br>            print(&quot;Chatbot:&quot;, bot_response)<br><br># Create an instance of the Chatbot class and start the conversation<br>chatbot = Chatbot()<br>chatbot.start_chatting()</pre><p>The functions defined above do the following:</p><p>Firstly, the model’s parameters, in a nutshell, do this:</p><blockquote><strong>Temperature</strong>: Controls the randomness of the responses. Higher values (e.g., 1.0) make the output more diverse, while lower values (e.g., 0.2) make it more focused and deterministic.</blockquote><blockquote><strong>Max Tokens</strong>: Limits the length of the response generated by the model.</blockquote><blockquote><strong>Top P</strong>: Specifies the cumulative probability threshold for choosing the next token. Higher values (e.g., 0.9) result in more focused responses.</blockquote><blockquote><strong>N</strong>: Determines the number of different responses generated by the model, which helps in exploring different possibilities.</blockquote><blockquote><strong>Stop</strong>: Allows us to specify a stopping phrase to indicate the end of the response.</blockquote><blockquote><strong>Frequency Penalty</strong>: Controls the model’s likelihood of repeating the same response.</blockquote><blockquote><strong>Presence Penalty</strong>: Controls how much the model considers using a token that hasn’t been mentioned in the conversation.</blockquote><p>Find more about fine-tuning these parameters<a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129"> here</a>.</p><p><strong>append_to_chat_history(message)</strong>: This function appends the user&#39;s message to the chat history stored in the chat_history list.</p><p><strong>read_personal_file(file_path)</strong>: This function utilizes the textractLibrary to extract text from personal files. It attempts to decode the extracted text using UTF-8 encoding. An error message is displayed if any errors occur during the extraction process.</p><p><strong>collect_user_data()</strong>: This function collects the user&#39;s data stored in the &quot;<strong>/data</strong>&quot; directory, placed inside the current working directory. It iterates through the files in the directory, determines their file types, and uses the appropriate method to extract text. It returns the combined user data as a string. The function glob.glob(os.path.join(data_directory, “*.*”))<br> is used to retrieve a list of file paths that match a specified pattern within a given directory. In this case *.*, which fits all files with any extension.</p><p><strong>create_chat_response(message)</strong>: This function constructs the chat response using the <strong>O</strong><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129"><strong>penAI ChatCompletion API</strong></a>. It appends the user&#39;s message and the collected user data (if any) to the message list. The API call is made with the provided messages, and the response is stored in the response variable. The function then appends the response to the chat history and returns it.</p><blockquote>More details about the OpenAI chat completion API, the model’s parameters, and fine-tuning the model’s parameters can be found<a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129"> here</a> on another blog post I wrote recently.</blockquote><p><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></p><p><strong>start_chatting()</strong>: This function initiates an interactive chat session with the user. It prompts the user for input, generates the bot&#39;s response using create_chat_response(), and prints the response. The conversation continues until the user enters &quot;exit&quot; to quit.</p><p>In the end the while True loop continuously prompts the user for input. <strong>To exit the chatbot, type &quot;exit.&quot;</strong></p><p>More details about the ChatGPT-3.5-turbo model and <a href="https://platform.openai.com/docs/api-reference">API docs</a> can be found below:</p><ul><li><a href="https://platform.openai.com/playground?mode=chat&amp;model=gpt-3.5-turbo-16k">OpenAI Platform</a></li><li><a href="https://platform.openai.com/docs/guides/gpt">OpenAI Platform</a></li></ul><h3>Step 5: Deploying the ChatGPT Bot</h3><p>To run the program, you must open a terminal and execute the Python file. In your terminal, run the following command:</p><pre>$ python chatGPTbot.py        <br><br>Or<br><br>$ python3 chatGPTbot.py</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zPr0VbyaewSuaxq8KEfh9A.png" /><figcaption>The model answers based on the data I stored inside /data directory</figcaption></figure><p><strong>Voila!</strong> Your personal ChatGPT bot is now ready to chat. Start interacting with it by entering messages; the bot will respond accordingly. When you’re finished, type “<strong>exit</strong>” to end the conversation.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FYTbZzCkRQCEJa%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FYTbZzCkRQCEJa%2Fgiphy.gif&amp;image=https%3A%2F%2Fi.giphy.com%2Fmedia%2FYTbZzCkRQCEJa%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="326" frameborder="0" scrolling="no"><a href="https://medium.com/media/333465f1a604a7ad52c02bba4683fd7d/href">https://medium.com/media/333465f1a604a7ad52c02bba4683fd7d/href</a></iframe><h3>Step 6: Customizing and Enhancing Your ChatGPT Bot</h3><ol><li><strong>Context and History:</strong> You can extend the chat history in the create_chat_response() function to include previous user-bot interactions. This allows the bot to have a better understanding of the conversation history.</li><li><strong>System Prompts</strong>: Experiment with different system prompts to influence the bot&#39;s behavior. For example, you can try using prompts like “<strong>Translate the following English text to French:</strong>” or “<strong>Help me with the following Python code:</strong>”</li><li><strong>User Instructions</strong>: Refine your user instructions to be more explicit and precise. Specify the desired format or ask the bot to think step-by-step before answering.</li><li>Experiment with different <a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">parameters</a>,<strong> such as </strong><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129"><strong>temperature and max tokens</strong></a><strong>, to customize the behavior of your chatbot.</strong></li></ol><p>More details about the above parameters and what values to set can be found <a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">here</a> in my other blog post.</p><p><a href="https://anishsinghwalia.medium.com/model-parameters-in-openai-api-161a5b1f8129">Model Parameters in OpenAI API</a></p><h3>Conclusion:</h3><p>Congratulations! You have learned how to create and deploy a powerful ChatGPT bot on your Mac/Linux machine using Python. The provided code allows your bot to consider and utilize personal user data from various file formats, enabling a more personalized user experience.</p><blockquote><strong>Feel free to customize further and enhance your bot’s capabilities. Now it’s time to unleash your creativity and build amazing chatbots. Do add in the comments section about your use-case and how you are using this personal chatGPT bot.</strong></blockquote><p><strong>NOTE:</strong></p><blockquote>Remember to handle personal data responsibly and comply with privacy regulations. <strong>Remember to secure your OpenAI API key and use it wisely</strong>.</blockquote><p>You can integrate it with other platforms, or <strong>build a web-based chatbot</strong>. With the versatility of ChatGPT and the simplicity of Python, the possibilities are endless.</p><p>Happy coding and happy chatting with your very own ChatGPT bot!</p><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow to get Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><p>You can find the GitHub repository for this project here on my <a href="https://github.com/anishsingh20/Personal-ChatGPT-Bot-For-Mac">GitHub</a> profile. Follow me on Medium and my <a href="https://github.com/anishsingh20">Github </a>to show some support and for some fantastic AI and ChatGPT-related content coming up.</p><p>Comment here if you have tried anything like this before and how you use ChatGPT to be more productive.</p><p><a href="https://github.com/anishsingh20/Personal-ChatGPT-Bot-For-Mac">GitHub - anishsingh20/Personal-ChatGPT-Bot-For-Mac: Create and deploy your own chatGPT bot written in Python using the ChatGPT API, LongChain and a few lines of code</a></p><p>Connect with me on my social profiles like <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> &amp; <a href="https://github.com/anishsingh20">Github</a>. And let’s collaborate.</p><p><a href="https://github.com/anishsingh20">anishsingh20 - Overview</a></p><p><strong>(Note: The code provided in this tutorial is just a starting point. Implementing appropriate error handling, security measures, and moderation systems is essential when deploying chatbots in real-world scenarios.)</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=951a16aaaff7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Types of Optimization Algorithms used in Neural Networks and Ways to Optimize Gradient Descent]]></title>
            <link>https://anishsinghwalia.medium.com/types-of-optimization-algorithms-used-in-neural-networks-and-ways-to-optimize-gradient-descent-1e32cdcbcf6c?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e32cdcbcf6c</guid>
            <category><![CDATA[optimization-algorithms]]></category>
            <category><![CDATA[activation-functions]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Wed, 10 Mar 2021 09:08:01 GMT</pubDate>
            <atom:updated>2024-01-11T13:30:17.051Z</atom:updated>
            <content:encoded><![CDATA[<p>Have you ever wondered which optimization algorithm to use for your Neural network Model to produce slightly better and faster results by updating the Model parameters such as <strong>Weights</strong> and <strong>Bias</strong> values? Should we use <strong>Gradient Descent</strong> or <strong>Stochastic</strong> gradient Descent or <strong>Adam?</strong></p><p>I too didn’t know about the major differences between these different types of Optimization Strategies and which one is better over another before writing this article.</p><blockquote>I’ll share this month’s bonus tip or best productivity tools that are cheap, effective, and a game changer, which I personally use, prefer, and insist you all try. So do check them out and use them.</blockquote><p><strong>Here is the Bonus tip for you all. Let’s take a pause here before diving into the blog post.</strong></p><p><strong>Sharing knowledge is the key to growth and success. I want to shed some light on the productivity tools that have transformed my daily life and that I use on a day-to-day basis. These tools are my absolute favorite, are a great value for money and have helped me achieve my goals. Do open and check out these tools.</strong></p><p><strong>NOTE:</strong></p><blockquote><strong>1) </strong><a href="https://affiliate.notion.so/sze1sb89euds"><strong>NOTION:</strong></a></blockquote><blockquote><strong>Bonus Tip 1</strong>: One great AI all-in-one Productivity/Task management tool I recently started using is<a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"> Notion</a>. Over the past few months, Notion has become famous and my absolute favorite.</blockquote><blockquote>If you’re like me, Juggling work, daily tasks, notes, and projects is tough. Multiple tabs for email, Slack, and Google Docs make it overwhelming. I personally use <a href="https://affiliate.notion.so/ty1pz2sw7nm6-4y5a7"><strong>Notion AI</strong></a>, which streamlines everything in one place. It’s a game-changer, and you won’t regret using it.</blockquote><blockquote>I’ve been using its <a href="https://affiliate.notion.so/pro"><strong>PRO</strong></a> version for a while now, and I must say, it’s been a complete game-changer for me. With almost every online co-working tool <a href="https://affiliate.notion.so/NotionIntegerations">integration</a> you can think of, it makes my daily work routine a breeze.</blockquote><blockquote>Plus, the <a href="https://affiliate.notion.so/pro"><strong>pricing</strong></a> is unbeatable/cheapest for the tonnes of features it provides compared to all other all-in-one AI productivity tools I have used. <strong>I have taken up the annual subscription for mere 8$/month. Another awesome tool which is litreally dirt cheap.</strong></blockquote><blockquote>I love its <a href="https://affiliate.notion.so/chromeextension">Web Clipper</a> and use its Mac app, and I also use Notion on my <a href="https://affiliate.notion.so/phoneapp">phone</a>. You can download the desktop app from <a href="https://affiliate.notion.so/downloadDesktop">here</a>.</blockquote><blockquote>Do check out this cool post about <a href="https://affiliate.notion.so/whatisnotion"><strong>Notion</strong></a> to know more about this brilliant platform.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wybnNoXH3ygPldUxJ0wZ6w.png" /><figcaption>Best all-in-one AI Productivity tool for this month</figcaption></figure><blockquote><strong>2)</strong><a href="https://try.quillbot.com/mimzuo3pn068"><strong>QUILLBOT:</strong></a></blockquote><blockquote><strong>Bonus Tip 2</strong>: One great AI Productivity Writing tool I recently started using for day-to-day writing and tasks such as plagiarism checker, grammar checker, <a href="https://try.quillbot.com/mimzuo3pn068">Quillbot-Flow</a> , paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $4/month(58 % off)</strong>. The price is literally dirt cheap compared to other writing and prductivity AI tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><blockquote>Coming back to the topic-</blockquote><h3><strong>What are Optimization Algorithms</strong></h3><p>Optimization algorithms help us to <strong><em>minimize (or maximize)</em></strong> an <strong>Objective </strong>function (<em>another name for </em><strong><em>Error</em></strong><em> function</em>) <strong>E(x) </strong>which is simply a mathematical function dependent on the Model’s internal <strong>learnable</strong> <strong>parameters</strong> which are used in computing the target values(<strong>Y</strong>) from the set of <em>predictors</em>(<strong>X</strong>) used in the model. For example — we call the <strong>Weights(W)</strong> and the <strong>Bias(b)</strong> values of the neural network as its internal learnable <em>parameters</em> which are used in computing the output values and are learned and updated in the direction of optimal solution i.e minimizing the <strong>Loss </strong>by the network’s training process and also play a major role in the <strong><em>training</em></strong> process of the Neural Network Model .</p><p><strong><em>The internal parameters of a Model play a very important role in efficiently and effectively training a Model and produce accurate results. This is why we use various Optimization strategies and algorithms to update and calculate appropriate and optimum values of such model’s parameters which influence our Model’s learning process and the output of a Model.</em></strong> <strong>Types of optimization algorithms</strong>?</p><p>Optimization Algorithm falls in 2 major categories -</p><p>1. <strong>First Order Optimization Algorithms </strong>— These algorithms minimize or maximize a Loss function <strong>E(x) </strong>using its <strong><em>Gradient</em></strong> values with respect to the parameters. Most widely used First order optimization algorithm is <strong>Gradient Descent.</strong>The First order derivative tells us whether the function is decreasing or increasing at a particular point. First order Derivative basically give us a <strong>line</strong> which is <strong><em>Tangential </em></strong><em>to a point on its Error Surface.</em></p><p><strong><em>What is the Gradient of a function?</em></strong>A <strong>Gradient</strong> is simply a vector which is a multi-variable generalization of a <strong><em>derivative</em></strong>(<strong>dy/dx</strong>) which is the <em>instantaneous rate of change of </em><strong><em>y with respect to x. </em></strong><em>The difference is that to calculate a derivative of a function which is dependent on more than one variable or multiple variables, a </em><strong><em>Gradient takes its place. And a gradient is calculated using Partial Derivatives . </em></strong><em>Also another major difference between the </em><strong><em>Gradient</em></strong><em> and a </em><strong><em>derivative</em></strong><em> is that a </em><strong><em>Gradient</em></strong><em> of a function produces a </em><strong><em>Vector Field</em></strong><em>.</em></p><p>A <strong>Gradient</strong> is represented by a <strong><em>Jacobian</em></strong> Matrix — which is simply a Matrix consisting of <strong><em>first-order partial Derivatives(Gradients)</em></strong>.</p><p>Hence summing up, a derivative is simply defined for a function dependent on single variables, whereas a Gradient is defined for a function dependent on multiple variables. Now let’s not get more into Calculus and Physics.</p><p><strong>2 . Second-Order Optimization Algorithms</strong> — Second-order methods use the <strong>second-order derivative</strong> which is also called <strong>Hessian</strong> to minimize or maximize</p><p>the<strong> Loss </strong>function. The <strong>Hessian</strong> is a Matrix of <strong><em>Second Order Partial Derivatives</em></strong>. <strong><em>Since the second derivative is costly to compute, the second-order is not used much</em></strong>.The second-order derivative tells us whether the <strong><em>first derivative</em></strong> is increasing or decreasing which hints at the function’s curvature. Second-Order Derivative provide us with a <strong>quadratic </strong>surface that touches the curvature of the <strong>Error Surface</strong>.</p><h3>Some Advantages of Second-Order Optimization over First Order —</h3><p>Although the Second Order Derivative may be a bit costly to find and calculate, the advantage of a <strong><em>Second-order</em></strong></p><p><strong><em>Optimization </em></strong><em>Technique is that does not neglect or ignore the </em><strong><em>curvature of the Surface</em></strong><em>. Secondly, in terms of Step-wise Performance, they are better.</em></p><p>You can search more on second order Optimization Algorithms here<em>https://web.stanford.edu/class/msande311/lecture13.pdf</em></p><h3><strong>So which Order Optimization Strategy to use?</strong></h3><p>1. Now <strong>The First Order Optimization</strong> techniques are easy to compute and less time consuming, converging pretty fast on large data sets.</p><p>2. <strong>Second Order Techniques</strong> are faster only when the <strong>Second Order Derivative</strong> is known otherwise, these methods are always slower and costly to compute in terms of both time and memory.</p><p><em>Although, sometimes </em><strong><em>Newton’s Second Order Optimization </em></strong><em>technique can sometimes Outperform </em><strong><em>First Order Gradient Descent </em></strong><em>Techniques because Second Order Techniques will not get stuck around paths of slow convergence around </em><strong><em>saddle points</em></strong><em> whereas </em><strong><em>Gradient Descent</em></strong><em> sometimes gets stuck and does not converges.</em></p><p>The best way to know which one converges fast is to try it out yourself.</p><h3><strong>Now, what are the different types of Optimization Algorithms used in Neural Networks?</strong></h3><h3>Gradient Descent</h3><p><strong><em>Gradient Descent</em></strong> is the most important technique and the foundation of how we train and optimize <strong><em>Intelligent Systems. </em></strong>What is does is <strong><em>—</em></strong></p><blockquote>“Oh Gradient Descent — Find the Minima, control the variance and then update the Model’s parameters and finally lead us to Convergence”</blockquote><p><strong>θ=θ−η</strong>⋅∇<strong>J(θ) </strong>— is the formula of the parameter updates, where ‘<strong>η’ </strong>is the learning rate ,’∇<strong>J(θ)’ </strong>is the<strong> Gradient</strong> of <strong><em>Loss function</em>(</strong>θ<strong>)<em> </em></strong>w.r.t <em>parameters-‘θ’</em>.</p><p>It is the most popular Optimization algorithms used in optimizing a Neural Network. Now gradient descent is majorly used to do <strong>Weights updates</strong> in a Neural Network Model , i.e update and tune the Model’s parameters in a direction so that we can minimize the <strong>Loss function</strong>. Now we all know a Neural Network trains via a famous technique called <strong>Backpropagation, </strong>in which we first propagate forward calculating the dot product of Inputs signals and their corresponding Weights and then apply a <strong><em>activation function </em></strong>to those sum of products, which transforms the input signal to an output signal and also is important to model complex Non-linear functions and introduces <strong>Non-linearities</strong> to the Model which enables the Model to learn almost any <em>arbitrary functional mappings.</em> After this we propagate <strong>backwards</strong> in the Network</p><p>carrying <strong>Error</strong> terms and updating <strong>Weights</strong> values using <em>Gradient Descent, in which we calculate the gradient of </em><strong><em>Error(E) function </em></strong><em>with respect to the </em><strong><em>Weights</em></strong><em> (</em><strong><em>W</em></strong><em>) or the parameters, and update the parameters (here </em><strong><em>Weights</em></strong><em>) in the opposite direction of the Gradient of the Loss function w.r.t to the Model’s parameters.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BSaMSrcdBk49b63GF0aFkQ.png" /><figcaption>Weight updates in the opposite direction of the Gradient- Source-Google.com</figcaption></figure><p><em>The image above shows the process of Weight updates in the opposite direction of the Gradient Vector of Error w.r.t to the Weights of the Network. The </em><strong><em>U-Shaped</em></strong><em> curve is the Gradient(slope). As one can notice if the Weight</em><strong><em>(W)</em></strong><em> values are too small or too large then we have large Errors, so want to update and optimize the weights such that it is neither too small nor too large, so we descent downwards opposite to the Gradients until we find local minima.</em></p><h3>Variants of Gradient Descent-</h3><p>The traditional<em> Batch Gradient Descent </em>will calculate the gradient of the whole Data set but <em>will perform only </em><strong><em>one update, hence it can be very slow and hard to control for datasets that are very very large and don’t fit in the Memory</em></strong><em>. </em>How big or small of an update to do is determined by the Learning Rate -<strong>η , </strong>and it<strong> </strong>is guaranteed to converge to the <strong><em>global minimum</em></strong><em> </em>for convex error surfaces and to a <strong><em>local minimum</em></strong> for non-convex surfaces<em>. Another thing while using Standard batch Gradient descent is that it computes redundant updates for large data sets.</em></p><p>The above problems of Standard Gradient Descent are rectified in Stochastic Gradient Descent.</p><h3>1. Stochastic gradient descent</h3><p>Stochastic Gradient Descent(SGD) on the other hand performs a parameter update for <strong>each training example</strong>.It is usually a much faster technique. It performs one update at a time.</p><p><strong>θ=θ−η</strong>⋅∇<strong>J(θ;x(i);y(i)) , where {x(i) ,y(i)} are the training examples</strong>. <em>Now due to these</em><strong><em> frequent updates</em></strong><em>, parameters updates have </em><strong><em>high variance</em></strong><em> and causes the </em><strong><em>Loss function to fluctuate to different intensities</em></strong><em>. This is actually a good thing because it helps us </em><strong><em>discover new and possibly better local minima</em></strong><em>, whereas Standard Gradient Descent will only converge to the minimum of the basin as mentioned above.</em></p><p><strong><em>But the problem with SGD is that due to the frequent updates and fluctuations it ultimately complicates the convergence to the exact minimum and will keep overshooting due to the frequent fluctuations.</em></strong></p><p>Although, it has been shown that as we slowly decrease the learning rate-<strong>η</strong>, SGD shows the same convergence pattern as Standard gradient descent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*4OTAd4-hHB8jbFp-3oF17w.png" /><figcaption>High Variance parameter updates for each training example cause the Loss function to fluctuate heavily due to which we might not get the minimum value of parameter which gives us least Loss value.</figcaption></figure><p><strong>The problems of high variance parameter updates and unstable convergence can be rectified in another variant called <em>Mini-Batch Gradient Descent</em>.</strong></p><h3>2. Mini Batch Gradient Descent</h3><p>An improvement to avoid all the problems and demerits of SGD and standard Gradient Descent would be to use <strong>Mini Batch Gradient Descent</strong> as it takes the best of both techniques and performs an update for every batch with n training examples in each batch.</p><h3>The advantages of using Mini Batch Gradient Descent are —</h3><p>1. It Reduces the variance in the parameter updates , which can ultimately lead us to a much better and stable convergence.</p><p>2. Can make use of highly optimized matrix optimizations common to state-of-the-art deep learning libraries that make computing the gradient w.r.t. a mini-batch very efficient.</p><p>3. Commonly Mini-batch sizes Range from 50 to 256, but can vary as per the application and problem being solved.</p><p>4. Mini-batch gradient descent is typically the algorithm of choice when training a neural network nowadays</p><p><strong>P.S — Actually the term SGD is used also when mini-batch gradient descent is used.</strong></p><h3>Challenges faced while using Gradient Descent and its variants —</h3><p>1. Choosing a proper learning rate can be difficult. A learning rate that is too small leads to painfully <em>slow convergence i.e </em>will result in <strong>small</strong> baby steps towards finding optimal parameter values which minimize loss and finding that valley which directly affects the overall training time which gets too large. While a learning rate that is too <strong>large</strong> can hinder convergence and cause the loss function to fluctuate around the minimum or even to diverge.</p><p>2. Additionally, the same learning rate applies to all parameter updates. If our data is sparse and our features have very different frequencies, we might not want to update all of them to the same extent, but perform a larger update for rarely occurring features.</p><p>3. Another key challenge of minimizing highly non-convex error functions common for neural networks is avoiding getting trapped in their numerous <strong><em>sub-optimal local minima</em></strong>. Actually, Difficulty arises in fact not from local minima but from <strong><em>saddle points</em></strong>, i.e. <em>points where one dimension slopes up and another slope down</em>. These saddle points are usually surrounded by a plateau of the same error, which makes it notoriously hard for</p><p>SGD to escape, as the gradient is close to zero in all dimensions.</p><h3>Optimizing the Gradient Descent</h3><p>Now we will discuss the various algorithms which are used to further optimize Gradient Descent.</p><h3>Momentum</h3><p><em>The high variance oscillations in SGD makes it hard to reach convergence</em> , so a technique called <strong><em>Momentum</em></strong> was invented which <strong><em>accelerates</em></strong> <strong>SGD</strong> by navigating along the relevant direction and softens the oscillations in irrelevant</p><p>directions.In other words all it does is adds a fraction ‘<strong>γ</strong>’ of the update vector of the past step to the current update vector.</p><p><strong>V(t)=γV(t−1)+η</strong></p><p><strong>J(θ)</strong>.</p><p>and finally we update parameters by <strong>θ=θ−V(t) .</strong></p><p>The momentum term <strong>γ</strong> is usually set to 0.9 or a similar value. <em>Here the </em><strong><em>momentum</em></strong><em> is same as the momentum in classical physics , as we throw a ball down a hill it gathers momentum and its velocity keeps on increasing.</em></p><p>The same thing happens with our parameter updates —</p><p>1. It leads to faster and stable convergence.</p><p>2. Reduced Oscillations</p><p>The <strong>momentum</strong> term <strong>γ</strong> increases for dimensions whose gradients point in the same directions and <em>reduces updates</em> for dimensions whose gradients change directions. <em>This means it does parameter updates only for relevant examples. This reduces the unnecessary parameter updates which lead to faster and stable convergence and reduced oscillations</em>.</p><h3>Nesterov accelerated gradient</h3><p>A researcher named Yurii Nesterov saw a problem with Momentum</p><p>However, a ball that rolls down a hill, blindly following the slope, is highly unsatisfactory. We’d like to have a smarter ball, a ball that has a notion of where it is going so that it knows to slow down before the hill slopes up again.</p><p>What actually happens is that as we reach the minima i.e the lowest point on the curve,the <strong>momentum</strong> is pretty high and it doesn’t know to<em> </em><strong><em>slow</em></strong><em> down</em> at that point due to the high momentum <em>which could cause it to miss the minima entirely and continue movie up. This problem was noticed by </em><strong>Yurii Nesterov.</strong></p><p>He published a research paper in 1983 which solved this problem of momentum and we now call this strategy <strong><em>Nesterov Accelerated Gradient.</em></strong></p><p><strong><em>In the method, he suggested we first make a big jump based on the previous momentum then calculate the Gradient and them make a correction which results in a parameter update. Now, this anticipatory update prevents us to go too fast and not miss the minima and makes it more responsive to changes.</em></strong></p><p><strong><em>Nesterov accelerated gradient (NAG)</em></strong> is a way to give our momentum term this kind of prescience. We know that we will use our momentum term <strong>γV(t−1)</strong> to move the parameters <strong>θ</strong>. Computing <strong>θ−γV(t−1)</strong> thus gives us an <em>approximation of the next position of the parameters</em> <em>which gives us a</em> <em>rough idea of where our parameters are going to be</em>. <strong>We can now effectively look ahead by calculating the gradient not w.r.t. to our current parameters θ but w.r.t. the approximate future position of our parameters:</strong></p><p><strong>V(t)=γV(t−1)+η</strong></p><p><strong>J( θ−γV(t−1) ) </strong>and then update the parameters using<strong> θ=θ−V(t) .</strong></p><p><em>One can refer more on </em><strong><em>NAGs</em></strong> here <a href="http://cs231n.github.io/neuralnetworks-3/">http://cs231n.github.io/neuralnetworks-3/.</a></p><p>Now that we are able to adapt our updates to the slope of our error function and speed up SGD in turn, we<em> would also like to adapt our updates to each individual parameter to perform larger or smaller updates depending on their importance.</em></p><h3>Adagrad</h3><p>It simply allows the learning Rate -<strong>η </strong>to <strong>adapt</strong> based on the parameters. So it makes big updates for infrequent parameters and small updates for frequent parameters. For this reason, it is well-suited for dealing with sparse data.</p><p><strong><em>It uses a different learning Rate for every parameter</em></strong><em> </em><strong><em>θ at a time step based on the past gradients which were computed for that parameter.</em></strong></p><p>Previously, we performed an update for all parameters <strong>θ</strong> at once as every parameter <strong>θ(i)</strong> used the same learning rate<strong> η</strong>. As <strong><em>Adagrad</em></strong> uses a different learning rate for every parameter <strong>θ(i)</strong> at every time step <strong>t</strong>, we first show Adagrad’s per-parameter update, which we then vectorize. For brevity, we set <strong>g(t,i)</strong> to be the <strong><em>gradient of the loss function </em></strong>w.r.t.</p><p>to the parameter<strong> θ(i)</strong> at time step <strong>t .</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/496/1*smGqiXNDqdTTdojrWXNdFg.png" /><figcaption>The formula for Parameter updates(Source: Google)</figcaption></figure><p><strong><em>Adagrad </em>modifies the general learning rate η at each time step t for every parameter θ(i) based on the past gradients that have been computed for θ(i).</strong></p><p><em>The main benefit of </em><strong><em>Adagrad</em></strong><em> is that we don’t need to manually tune the learning rate.</em> <em>Most implementations use a default value of 0.01 and leave it at that.</em></p><h3>Disadvantage —</h3><p>1. Its main weakness is that its learning rate-<strong>η</strong> is always Decreasing and decaying.</p><p>This happens due to the accumulation of each squared Gradients in the denominator since every added term is positive. The accumulated sum keeps growing during training. This in turn causes the <em>learning rate to shrink and eventually become so small,</em> <strong>that the model just stops learning entirely and stops acquiring new additional knowledge. Because we know that as the learning rate gets smaller and smaller the ability of the Model to learn fastly decreases and which gives very slow convergence and takes very long to train and learn i.e learning speed suffers and decreases.</strong> This problem of <strong>Decaying learning Rate</strong> is Rectified in another algorithm called <strong>AdaDelta.</strong></p><h3>AdaDelta</h3><p>It is an extension of <strong>AdaGrad</strong> which tends to remove the <em>decaying learning Rate</em> problem of it. Instead of accumulating all previously squared gradients, <strong><em>Adadelta</em></strong> limits the window of accumulated past gradients to some fixed size <strong>w</strong>.</p><p>Instead of inefficiently storing <strong>w</strong> previous squared gradients, the sum of gradients is recursively defined as a <em>decaying </em><strong><em>mean</em></strong> of all past squared gradients. The running average <strong>E[g²](t)</strong> at time step <strong>t</strong> then depends (as a fraction <strong>γ</strong> similarly to the Momentum term) <em>only on the previous average and the current gradient.</em></p><p><strong>E[g²](t)=γ.E[g²](t−1)+(1−γ).g²(t) , </strong>We set <strong>γ</strong> to a similar value as the momentum term, around 0.9.</p><p>Δθ(t)=−η</p><p>g(t,i). θ(t+1)=θ(t)+Δθ(t).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/805/1*brufR2JEH-KziQAhtxopsQ.png" /><figcaption><strong>The final formula for Parameter Updates(Source:google)</strong></figcaption></figure><p><strong>Another thing with AdaDelta is that we don’t even need to set a default learning Rate</strong></p><h3>What Improvements we have done so far</h3><p>1. We are calculating <em>different learning Rates</em> for each parameter.</p><p>2. We are also calculating<em> momentum</em>.</p><p>3. Preventing <strong>Vanishing(decaying) learning Rates</strong>.</p><p>What more improvements can we do ?</p><p>Since we are calculating individual<strong><em> learning rates</em></strong> for each parameter , why not calculate individual <strong>momentum</strong> changes for each parameter and store them separately. This is where a new modified technique and improvement comes into play called as <strong>Adam.</strong></p><h3>Adam</h3><p>Adam stands for <strong>Adaptive Moment Estimation.</strong> Adaptive Moment Estimation (Adam) is another method that computes adaptive learning rates for each parameter. In addition to storing an exponentially decaying average of past squared gradients like <strong>AdaDelta</strong> ,<strong><em>Adam</em></strong><em> also keeps an exponentially decaying average of past gradients M</em><strong><em>(t)</em></strong><em>, similar to momentum</em>:</p><p><strong>M(t) and V(t)</strong> are values of the first moment which is the <strong><em>Mean</em></strong> and the second moment which is the <strong><em>uncentered variance</em></strong> of the <em>gradients</em> respectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*W3NMTJao5xADBmbq5yKqwg.png" /><figcaption><strong>The formulas for the first Moment(mean) and the second moment (the variance) of the Gradients</strong></figcaption></figure><p>Then the final formula for the Parameter update is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/462/1*ZxigpkZ7j0K0Xa2dN5v5jA.png" /></figure><p>The values for β1 is 0.9 , 0.999 for β2, and (10 x exp(-8)) for <strong>ϵ</strong>. <strong><em>Adam</em></strong><em> works well in practice and compares favorably to other adaptive learning-method algorithms as it converges very fast and the learning speed of the Model is quiet Fast and efficient and also it rectifies every problem that is faced in other optimization techniques such as </em><strong><em>vanishing Learning rate, slow convergence or High variance in the parameter updates which leads to fluctuating Loss function</em></strong></p><h3>Conclusion</h3><p><strong>Which optimizer should we use?</strong></p><p>The question was to choose the best optimizer for our Neural Network Model in order to converge fast and to learn properly and tune the internal parameters so as to minimize the Loss function.</p><p><strong>Adam </strong><em>works well in practice and outperforms other Adaptive techniques.</em></p><p>If your input data is sparse then methods such as <strong>SGD, NAG and momentum</strong> are inferior and perform poorly. <strong>For sparse data sets, one should use one of the <em>adaptive learning-rate</em> methods</strong>. An additional benefit is that we won’t need to adjust the learning rate but likely achieve the best results with the default value.</p><p>If one wants fast convergence and train a Deep Neural Network Model or a highly complex Neural Network then <strong>Adam or any other Adaptive learning rate techniques </strong>should be used because they outperform every other optimization algorithms.</p><p>I hope you guys liked the article and were able to give you a good intuition towards the different behaviours of different Optimization Algorithms.</p><p>Please <a href="https://medium.com/@anishsingh20/subscribe">Subscribe</a> and Follow to get Free access to my newsletter and keep yourself updated on the latest AI and ChatGPT trends and technologies to make your lives easier and more productive, save money, and be effective at whatever you do.</p><p><a href="https://medium.com/@anishsingh20/subscribe">Get notified whenever I publish a new story to stay up to date on the latest AI &amp; ChatGPT trends.</a></p><p>Your support motivates me to keep researching, designing cheatsheets, and writing about such topics.</p><h3>References —</h3><p>1. Optimizing Gradient Descent- <a href="http://sebastianruder.com/optimizing-gradient-descent/">http://sebastianruder.com/optimizing-gradient-descent/</a></p><p>2. Dean, J., Corrado, G. S., Monga, R., Chen, K., Devin, M., Le, Q.</p><p>V, … Ng, A. Y. (2012). Large Scale Distributed Deep Networks.</p><p>NIPS 2012: Neural Information Processing</p><p>Systems. <a href="http://doi.org/10.1109/ICDAR.2011.95">http://doi.org/10.1109/ICDAR.2011.95</a></p><p>3. Ioffe, S., &amp;amp; Szegedy, C. (2015). Batch Normalization : Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv Preprint arXiv:1502.03167v3.</p><p>4. Qian, N. (1999). On the momentum term in gradient descent learning algorithms. Neural Networks : The Official Journal of the</p><p>International Neural Network Society, 12(1), 145–</p><p>151. <a href="http://doi.org/10.1016/S0893-6080(98)00116-6">http://doi.org/10.1016/S0893-6080(98)00116-6</a></p><p>5. Kingma, D. P., &amp;amp; Ba, J. L. (2015). Adam: a Method for</p><p>Stochastic Optimization. International Conference on Learning Representations</p><p>6. Zaremba, W., &amp;amp; Sutskever, I. (2014). Learning to Execute, 1–</p><p>25. Retrieved from <a href="http://arxiv.org/abs/1410.4615">http://arxiv.org/abs/1410.4615</a></p><p>7. Zhang, S., Choromanska, A., &amp;amp; LeCun, Y.</p><p>(2015). Deep learning with Elastic Averaging SGD. Neural</p><p>Information Processing Systems Conference (NIPS</p><p>2015).Retrieved from <a href="http://arxiv.org/abs/1412.6651">http://arxiv.org/abs/1412.6651</a></p><p>8. Darken, C., Chang, J., &amp;amp; Moody, J. (1992). Learning rate schedules for faster stochastic gradient search. Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop,</p><p>(September). <a href="http://doi.org/10.1109/NNSP.1992.253713">http://doi.org/10.1109/NNSP.1992.253713</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e32cdcbcf6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Activation functions used in Neural Networks- Which is better ?]]></title>
            <link>https://anishsinghwalia.medium.com/activation-functions-used-in-neural-networks-which-is-better-d4817d91199a?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4817d91199a</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[datacamp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[activation-functions]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Sat, 31 Oct 2020 13:40:21 GMT</pubDate>
            <atom:updated>2023-12-14T11:42:04.468Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Activation functions used in Neural Networks-Which is better?</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*w1Y8asUM669QogAtKU6a9A.png" /><figcaption><strong>Activation Functions(Source:Google Images)</strong></figcaption></figure><h4><strong>What are Artificial Neural Networks?</strong></h4><p>Now, I know we all are familiar with what A-NN is but still let me define it for my own satisfaction -It is a very powerful, strong as well as a very complicated Machine Learning technique which mimics a human brain and how it functions.</p><p>Like our human brain has billions of neurons in a hierarchy and Network of <strong>neurons</strong> which are interconnected with each other via <strong>Axons</strong> and passes Electrical signals from one layer to another called <strong>synapses. </strong>This is how we humans learn things. Whenever we see, hear, feel and think something a <strong>synapse(electrical impulse)</strong> is fired from one neuron to another in the hierarchy which enables us to learn, remember and memorize things in our daily life since the day we were born.</p><p>Okay, now let’s not get into Biology.</p><blockquote><strong>Bonus Tip</strong>: One great tool I recently started using for writing and tasks such as plagiarism checker, grammar checker, Co-writer, paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $2/month (45% off) during the Year-end sale using coupon code — (HOLIDAY45),</strong> valid till <strong>December end</strong>. The price was literally dirt cheap compared to other writing tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><h3><strong>What are Activation functions and what are it uses in a Neural Network Model?</strong></h3><p><strong><em>Activation</em></strong><em> functions</em> are really important for an Artificial Neural Network to learn and make sense of something really complicated and Non-linear complex functional mappings between the inputs and response variable. <em>They</em> <em>introduce non-linear properties to our Network</em>. <strong><em>Their main purpose is to convert an input signal of a node in an A-NN to an output signal.</em></strong> That output signal now is used as an input in the next layer in the stack.</p><p>Specifically in A-NN we do the sum of products of inputs(<strong>X</strong>) and their corresponding Weights(<strong>W</strong>) and apply an Activation function <strong>f(x)</strong> to it to get the output of that layer and feed it as an input to the next layer.</p><h4><strong>The question arises that why can’t we do it without activating the input signal?</strong></h4><p>If we do not apply an Activation function then the output signal would simply be a simple <strong><em>linear function</em></strong><em>. </em>A <em>linear function</em> is just a polynomial of <strong>one degree. </strong>Now, a linear equation is easy to solve but they are limited in their complexity and have less power to learn complex functional mappings from data. A Neural Network without an Activation function would simply be a <strong>Linear regression model, </strong>which has limited power and does not perform well most of the time. We want our Neural Network to not just learn and compute a linear function but something more complicated than that. <em>Also without activation function, our Neural network would not be able to learn and model other complicated kinds of data such as images, videos, audio, speech etc. That is why we use Artificial Neural network techniques such as</em><strong><em> Deep learning to make sense of something complicated, high dimensional, non-linear -big datasets, where the model has lots and lots of hidden layers in between and has a very complicated architecture which helps us to make sense and extract knowledge from such complicated big datasets.</em></strong></p><h3><strong>So why do we need Non-Linearities?</strong></h3><p>Non-linear functions are those which have a degree more than one and they have a curvature when we plot a Non-Linear function. Now we need a Neural Network Model to learn and represent almost anything and any arbitrary complex function which maps inputs to outputs. Neural-Networks are considered <strong><em>Universal Function Approximators</em></strong>. <em>It means that they can compute and learn any function at all</em>. Almost any process we can think of can be represented as a functional computation in Neural Networks.</p><p><em>Hence it all comes down to this, we need to apply an activation function f(x) so as to make the network more powerful and add the ability to it to learn something complex and complicated form data and represent non-linear complex arbitrary functional mappings between inputs and outputs. Hence using a non-linear Activation we are able to generate non-linear mappings from inputs to outputs.</em></p><p><strong>Also, another important feature of an Activation function is that it should be differentiable. We need it to be this way so as to perform backpropagation optimization strategy while propagating backwards in the network to compute gradients of Error(loss) with respect to Weights and then accordingly optimize weights using Gradient descend or any other Optimization technique to reduce Error.</strong></p><p>Just always remember to do :</p><blockquote><strong><em>“</em></strong><em>Input times weights, add Bias and Activate”</em></blockquote><h3><strong>Most popular types of Activation functions -</strong></h3><p>1. <em>Sigmoid or Logistic</em></p><p>2. <em>Tanh — Hyperbolic tangent</em></p><p>3. <em>ReLu -Rectified linear units</em></p><p><strong>Sigmoid Activation function: </strong>It is a activation function of form <strong>f(x) = 1 / 1 + exp(-x) . </strong>Its Range is between 0 and 1. It is a S-shaped curve. It is easy to understand and apply but it has major reasons which have made it fall out of popularity -</p><ul><li>Vanishing gradient problem</li><li>Secondly, its output isn’t zero centred. It makes the gradient updates go too far in different directions. <strong>0 &lt; output &lt; 1, and it makes optimization harder.</strong></li><li>Sigmoids saturate and kill gradients.</li><li>Sigmoids have slow convergence.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*pAiWanKO-zLOlp6b3_yvoQ.png" /><figcaption>Sigmoid Function(Source: google.com)</figcaption></figure><p>Now how do we solve the above problems?</p><p><strong><em>Hyperbolic Tangent function- Tanh : </em></strong>It’s mathamatical formula is <strong>f(x) = 1 — exp(-2x) / 1 + exp(-2x). </strong>Now it’s output is zero centered because its range in between -1 to 1 i.e -1 &lt; output &lt; 1 . Hence optimization is <em>easier</em> in this method hence in practice it is always preferred over Sigmoid function . <em>But still it suffers from </em><strong><em>Vanishing gradient problem</em></strong><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*fSUyXaZ5O-vmYS1kVs8_xA.png" /><figcaption>TanH function(Source:google.com)</figcaption></figure><h4>Then how do we deal and rectify the vanishing gradient problem?</h4><p><strong><em>ReLu- Rectified Linear units</em></strong>: It has become very popular in the past couple of years. It was recently proved that it had <em>6 times improvement in convergence</em> from Tanh function. It’s just <strong><em>R(x) = max(0,x) i.e if x &lt; 0 , R(x) = 0 and if x &gt;= 0 , R(x) = x.</em></strong><em> Hence as seeing the mathematical form of this function we can see that it is very simple and efficient. A lot of times in Machine learning and computer science we notice that most simple and consistent techniques and methods are only preferred and are best. </em>Hence it avoids and rectifies the <strong>vanishing gradient</strong> problem. Almost all deep learning Models use <strong>ReLu</strong> nowadays.</p><p><strong>But its limitation is that it should only be used within Hidden layers of a Neural Network Model</strong>.</p><p>Hence for output layers, we should use a <strong>Softmax</strong> function for a Classification problem to compute the probabilities for the classes, and for a regression problem, it should simply use a <strong>linear</strong> function.</p><p><strong>Another problem with ReLu is that some gradients can be fragile during training and can die. It can cause a weight update which will make it never activate on any data point again. Simply saying that ReLu could result in Dead Neurons.</strong></p><p>To fix this problem another modification was introduced called <strong><em>Leaky ReLu </em></strong>to fix the problem of dying neurons. It introduces a small slope to keep the updates alive.</p><p>We then have another variant made form both ReLu and Leaky ReLu called <strong>Maxout </strong>function<strong>.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*92xcbxOLS77baKRrNGSWoQ.png" /><figcaption>ReLu(Source: google.com)</figcaption></figure><p>Enough theory right? , then why not go and compare the different activation functions and their performance yourself. Pick up a simple dataset and implement deep learning on it and try different activation function at various times. You will see the difference by yourself.</p><p>And for all the R lovers if you want to implement deep learning in R you can visit my <a href="https://github.com/anishsingh20/Deep-Learning-in-R-using-Keras-and-Tensorflow-"><strong><em>GitHub repository</em></strong><em> </em></a>on making a simple <em>digit recognizer on MNIST dataset </em>and use it as a reference to make deep learning models in R using <em>Keras</em> package for R.<br> <br> You can connect with me on <a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a> and <a href="https://github.com/anishsingh20">Github</a> and reach out to me in case you have any doubts or suggestions.</p><h3><strong>Conclusion</strong></h3><p><em>The question was which one is better to use?</em></p><p>Answer to this question is that nowadays we should use <strong>ReLu </strong>which should only be applied to the hidden layers. And if your model suffers from dead neurons during training we should use <strong>leaky ReLu</strong> or <strong>Maxout</strong> function.</p><p>It’s just that <em>Sigmoid and Tanh </em>should not be used nowadays due to the <strong>vanishing Gradient Problem</strong> which causes a lot of problems to train, degrades the accuracy and performance of a <strong>Deep Neural Network Model.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4817d91199a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Developing Web Apps in R using Shiny]]></title>
            <link>https://anishsinghwalia.medium.com/developing-web-apps-in-r-using-shiny-f868f51a4122?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/f868f51a4122</guid>
            <category><![CDATA[shiny]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[datacamp]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Wed, 27 May 2020 14:06:26 GMT</pubDate>
            <atom:updated>2023-12-14T11:43:01.157Z</atom:updated>
            <content:encoded><![CDATA[<p>In this tutorial, you will learn about the purpose of Shiny package in R, it’s fundamentals and how to develop a simple web application/dashboard in R. At the same time, you’ll also get introduced to a javascript based visualization package called <a href="http://jkunst.com/highcharter/highcharts.html"><strong>Highcharter</strong></a>.</p><p>All the links to web apps developed by me mentioned in this article extensively use the above package <a href="http://jkunst.com/highcharter/highcharts.html"><strong>Highcharter</strong></a><strong>. </strong>It is very interactive and easy to use. You will know how easy and simple it is to develop a web app in R and how powerful it is.</p><p>In my opinion in Data Science and Analytics, it is really important to express what you have analysed and share your findings in the most subtle yet visually attractive way possible and building web apps, beautiful interactive dashboards are the best way to do that. This article will make you competent in exactly that if you know R and or still learning it. This article will let you expand your skillset and move a step higher and better in becoming a better data professional. <em>I will add quite a few links to dashboards and web apps developed by me in this article which you can surely use for your reference and develop your own web applications</em>.</p><blockquote><strong>Bonus Tip</strong>: One great tool I recently started using for writing and tasks such as plagiarism checker, grammar checker, Co-writer, paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $2/month (45% off) during the Year-end sale using coupon code — (HOLIDAY45),</strong> valid till <strong>December end</strong>. The price was literally dirt cheap compared to other writing tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><blockquote><strong><em>SIDE NOTE:</em></strong></blockquote><blockquote>I always make sure I share my knowledge and resources which helped me. So I will be adding a few links to my favourite courses below related to this post and which can help you experiment with this post on your own<em>. My focus has always been to learn things in a practical manner, that is why </em><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong><em> </em></a><em>has always been my go-to place for learning from the very start. It’s just that in this manner you can actually learn faster and better. </em><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a><em> also has an amazing library of Projects. They started this new service recently and it’s really amazing. </em><a href="https://datacamp.pxf.io/MXNran"><strong>DataCamp projects</strong></a><strong> </strong><em>is another amazing thing which I keep doing from time to time. Make sure to give it a try. Just once try them and then you will understand why I love </em><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a><em> so much.</em></blockquote><p><strong>DataCamp will be offering a first-month subscription for only $1 from today till 3rd Dec, 8 PM ET. Also, they are having a black Friday sale where Yearly subscriptions have 75 % off. This is the first time they have ever run a sale like this. So do make sure to go check out </strong><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a><strong>. Also, </strong><a href="https://datacamp.pxf.io/x9N7JA"><strong><em>DATCAMP CHEATSHEETS</em></strong></a><strong> is a pretty handy and very useful tool which I use from time to time.</strong></p><blockquote>Firstly, If you are not familiar with the basics of the R language, I urge readers to go complete this amazing course by<a href="https://www.datacamp.com/courses?tap_a=5644-dce66f&amp;tap_s=210732-9d6bbf"><strong> </strong></a><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a> on <a href="https://datacamp.pxf.io/QO6kV6"><strong>R Programming</strong></a>, and this is the course from where I started my Data science journey using R years ago. Trust me with this, these courses are worth your time and money or if you want to learn the complete data science fundamentals from scratch using R which includes Statistics and probability, Data viz, exploring the data, data modelling and machine learning and other important concepts, I strongly recommend this foundation course <a href="https://datacamp.pxf.io/QO6kV6"><strong>Data Science with R</strong></a> .</blockquote><blockquote><a href="https://datacamp.pxf.io/5b79gD"><strong><em>Building dashbaords with ShinyDashboard</em></strong></a><strong><em>.</em></strong></blockquote><blockquote><a href="https://datacamp.pxf.io/AoEJWD"><strong>Building web apps with Shiny in R: Case Study</strong></a></blockquote><blockquote><a href="https://datacamp.pxf.io/3PdDJA"><strong>Intermediate interactive data visualization with Plotly in R</strong></a></blockquote><blockquote><a href="https://datacamp.pxf.io/4eWQGG"><strong>Building Dashboards with FlexDashboards</strong></a></blockquote><blockquote><a href="https://datacamp.pxf.io/7mJjaO"><strong><em>Building web apps in R using Shiny.</em></strong></a></blockquote><blockquote><a href="https://datacamp.pxf.io/do7myM"><strong>Building Chatbots in Python</strong></a></blockquote><blockquote><a href="https://datacamp.pxf.io/kjoBOv"><strong>Building Recommendation engines using PySpark</strong></a></blockquote><blockquote>So this would literally be the best time to grab some yearly subscriptions(<strong>which I have</strong>) which basically has unlimited access to all the courses and other things on <a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a> and make fruitful use of your time sitting at home during this Pandemic. So go for it folks and Happy learning, make the best use of this quarantine time and come out of this pandemic stronger and more skilled.</blockquote><h3>We’ll cover the following topics:</h3><ol><li>Introduction to <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-intro">shiny</a>.</li><li>How shiny works and the <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-funda">fundamentals of </a><a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-funda">shiny</a>.</li><li>How to <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-ui">design the UI</a> of a shiny application.</li><li>How to develop the <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-server">Server-side logic</a>.</li><li>A simple example <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-example">of a </a><a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-example">shiny app</a>.</li><li>Use Case — <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-use-case">Analyzing kaggle 2017 datascience survey data using </a><a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-use-case"><em>Highcharter</em></a> .</li><li>Deploying your <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-section-deploying-your-application">shiny app.</a></li></ol><p>Do read it till the end, because I have explained developing a basic dashboard with an example at the end, so you can use that as a reference and experiment with it.</p><p>Also below are some of the dashboards I have developed over the years in R: You can use these as a reference to build apps too-</p><ol><li><a href="https://github.com/anishsingh20/Analyzing-Corona-Virus-Spread"><strong>Worldwide COVID-19 Tracker</strong></a></li><li><a href="https://github.com/anishsingh20/COVID-19-INDIA-TRACKER"><strong>COVID-19 India Tracker</strong></a></li><li><a href="https://github.com/anishsingh20/Analysis-of-Indian-Economy-Using-Shiny"><strong>Analysis of Indian Economy using Shiny</strong></a></li><li><a href="https://github.com/anishsingh20/Kaggle-Data-science-survey-data-analysis-Using-Shiny"><strong>Kaggle Data Science Survey data analysis app</strong></a><strong>(The use case explained at the end of this article)</strong></li><li><a href="https://github.com/anishsingh20/Text-message-classification-shiny-web-app"><strong>Text message classification app</strong></a><strong>(Spam or Ham).</strong></li><li><a href="https://github.com/anishsingh20/Analzying-Inflation-Rates-Worldwide"><strong>Analyzing Inflation Rates Worldwide dashboard</strong></a></li></ol><p><a href="https://datascienceplus.com/making-a-shiny-dashboard-using-highcharter-analyzing-inflation-rates/"><em>Here</em></a> is another tutorial I wrote on using <a href="https://datascienceplus.com/making-a-shiny-dashboard-using-highcharter-analyzing-inflation-rates/">HighCharter</a> and developing dashboards in R.</p><p><a href="https://datascienceplus.com/kaggle-data-science-survey-data-analysis-using-highcharter/"><em>Here</em></a> is another one where I have used HighCharter for analyzing the dataset I have used for the app explained in this article.</p><p><strong><em>The above links will take you to the Github repositories of these apps, the deployed links to these apps are on their Github repo(s). So make sure to check them out and Fork them.</em></strong></p><p>So let’s get started.</p><h3>Introduction</h3><p>You might have already heard about shiny, a well-known R package that is actually a web application development framework for R. You can use it to build interactive web applications, dashboards and so much more, like deploying your trained machine learning model as a web service etc. All of these things make this package provides an amazing and powerful web framework for R users to develop beautiful interactive dashboards, web applications and turn your analysis into interactive web applications.</p><p><strong><em>One of the best things about </em></strong><strong><em>shiny is that it does not require you to have prior knowledge of HTML, CSS or javascript to develop web applications.</em></strong> It is all builtin and has several functions to design a beautiful User Interface (UI) for your application. UI for the applications can be built completely using R or can be written directly in HTML, CSS, and JavaScript for more flexibility. The CSS framework which is used by shiny UI is based on Twitter Bootstrap. Shiny has pre-built output widgets for displaying plots, tables, and printed output of R objects. Outputs will change simultaneously as users modify inputs, without reloading the browser in an <strong><em>asynchronous</em></strong> manner. For example, the way <strong>AJAX</strong> works. Shiny uses a <strong>reactive programming</strong> model. After the development of the application, next is the task of deploying your shiny application. You can easily deploy the shiny app from your Rstudio IDE in minutes. It Requires only an account on <a href="http://www.shinyapps.io/">http://www.shinyapps.io/</a>.</p><h3>Getting Started with Shiny</h3><p>Now that you know that Shiny combines the computational power of R with the interactivity of the modern web so that you can build standalone web applications, dashboard, interactive documents and reports using R Markdown, and gadgets with this R package, it’s time to get your hands dirty yourself.</p><h3>Fundamentals</h3><p>Shiny is available on CRAN and you can download it with the install.packages() function in R like below:</p><pre>install.packages(“shiny”)</pre><p>After you have done that, you can load in the library and start by launching and checking out a Shiny example application, such as &quot;01_hello&quot;:</p><p>runExample(&quot;01_hello&quot;)</p><p>The result of these lines of code makes it clear that Shiny applications have two separate components: a User interface and a Server script.</p><ol><li>The <strong><em>user interface</em></strong> is where you will design the layout of your application example, where to place the widgets, input elements such as textbox, slider, buttons, form elements and where to display the plots etc that is the frontend of your app.</li></ol><p>2. The<strong> <em>server</em> script</strong> will consist of the serverside logic which will contain all under the hood R code to make plots, to reactively render plots on some user-based event, build outputs based on the user inputs in clientside dynamically etc and other logic.</p><p>There is<strong> <em>bidirectional</em></strong> communication between the server and the UI to help each other. It can be mapped to a simple client/server communication architecture, where the server responds to the client’s requested data and resources. In Shiny, all this happens at a lightning-fast speed. This is because Shiny uses a <strong><em>Reactive programming</em></strong> paradigm. The inputs defined are reactive in nature which produce reactive outputs.</p><p>But what is the reactive programming paradigm exactly?</p><h3>Reactive Programming: A Short Definition</h3><p>Reactive programming is an <em>asynchronou</em>s programming model which is concerned with data streams and the dynamic changes in the program. Reactive programs continuously interact with their environment. Reactive programs only work in response to external changes and events to mostly deal with accurate interrupt handling. Real-time programs are usually reactive. Today, most of the web frameworks follow a Reactive programming model. For example, Django framework, AngularJs, MVC model, view, controller-based web frameworks are the best examples of reactive programming. Reactive programming can facilitate changes in an underlying model which is automatically reflected in a view.</p><p>Shiny also uses reactive programming paradigm in the same way. The UI and the server are continuously in contact with each other, and whenever an event or change in input occurs, shiny keeps track of your R code to be re-executed, and the changes are automatically/reactively reflected and updated in the output. Reactivity automatically occurs whenever you use an input value to render an output object.</p><h3>Shiny Inputs and Outputs Objects</h3><p>The shortest possible single file shiny app template:</p><pre><strong># app.R</strong></pre><pre><strong>library</strong>(shiny) <em>#loading the shiny package</em><br><br>ui &lt;- fluidPage() <em>#Object contains the UI of the app</em><br><br>server &lt;- <strong>function</strong>(input, output) {} <em>#server logic</em><br><br><em># both UI and server code in same file</em><br><br>shinyApp(ui = ui, server = server) <em>#running the app</em></pre><p>The above is an example of UI and server-side logic in the same file i.e <strong>app.R</strong>.</p><p>In the above code, the function attached to the ui that is fluidPage() is a function for creating fluid page layouts. A fluid page layout consists of rows which in turn include columns. This is same as bootstrap(a popular frontend CSS framework). You will add elements to the UI as parameters to the fluidPage function.</p><pre>ui &lt;- fluidPage(</pre><pre>#UI of the app is defined here</pre><pre>  <em># *Input() objects</em><br>  <em># *Output() objects</em><br>)</pre><p>The complete shiny application is built around the input and the output elements. <strong>Input</strong> can be anything ranging from a select input, checkboxes, radio buttons, text field, button, sliders, form elements etc.<strong> Outputs</strong> can be plots, tables, some text etc which are defined and changes reactively as the input value changes etc.</p><p>You will learn how to add elements to the UI and telling the server how to assemble the inputs and outputs in the below sections in more detail.</p><p>In Shiny, you can build and maintain your app with the ui.R file and server.R script separately, or you can build the entire app in a single file app.R like shown above. It depends on the size and complexity of your application. For large complex applications, you should prefer the modular way of keeping everything separate like shown below. Keeping UI and server files separate will make it easier to manage the code and application.</p><p>Now to start building the shiny application in R, you just need to open up you Rstudio IDE, go to File -&gt; New File -&gt; Shiny Web App and this is it. RStudio will first ask you the details of your application and there you can select whether you want a single file app or multiple files app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x3wc7FXpvWRW2Vj4NP1ydA.png" /><figcaption>Create a. new Shiny application in RStudio IDE</figcaption></figure><p>Now deploying a single file shiny app and a multiple file shiny app is exactly same and simple. You can simply deploy it from your RStudio IDE which will be explained later in the last section of this tutorial on <strong>Deploying your app</strong>.</p><p>To run your application, if your application resides in a directory ShinyApp/, you can launch it with:</p><p>runApp(&quot;ShinyApp&quot;) function with the directory name of the app as an argument.</p><h3>ui.R</h3><ol><li>*Inputs()- create an input using this function. Every *Input(inputId,label,....) the function takes the first parameter as an input id, using which it’s value will be accessed in the server script to add reactivity. Add the type of input you want and prepend it to *Input, for example- To make a slider input sliderInputs(), to make a select Input selectInput() etc. There are lots of input functions such as checkboxInput(), dateInput(), fileInput(),numericInput(), passwordInput(), textInput(), radioButtons() etc.</li></ol><p>For example:</p><pre><strong>library</strong>(shiny)<br></pre><pre>sliderInput(inputId = &quot;slider&quot;,<br>            label = &quot;Select a number&quot;,<br>            value = 10, <br>            min = 0, <br>            max = 100 )</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/714/1*sL5djpBNnirt3CRsz39jlw.png" /><figcaption>Slider Input</figcaption></figure><p>The above is what is produced from the above slider Input.</p><pre>selectInput(inputId=&quot;selctid&quot;,label=&quot;Select&quot;,choices=c(&quot;R&quot;,&quot;Python&quot;,&quot;Tensorflow&quot;))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*7N-Cym7zNmvd8dytkzBMZw.png" /><figcaption>Select Box</figcaption></figure><pre><em>#use ?selectInputs to read the documentation and explore more about using these input functions</em></pre><p>This is how you will create a reactive input in the UI of your application.</p><p>2. *Output()- use this function in the ui.R file to generate reactive outputs and view them in the app. Function takes output id as a parameter *Output(id). This output id will be used in the server script to write the R logic to display the type of reactive output you want. Prepend the type of output you want to the *Output() as, example-PlotOutput(),imageOutput(),tableOutput(),textOutput() etc are some of the most used output functions. example-plotOutput(&quot;scatterPlot&quot;) is how you will declare an output in the UI.</p><p>A simple demonstration- A general ui.r file would look like this. When using separate files you can use shinyUI() the function to start with designing the UI.</p><p>Divide the UI into 3 parts-</p><ol><li>Header — Use headerPanel() to add the app title.</li><li>Sidebar Section — consists of the widgets and the input variables to control the output. Use sidebarPanel( *Inputs() ) to add input objects to the sidebar.</li><li>Main Section — display the outputs such as plots, text etc in this section. Use mainPanel( *Output() objects) to define output objects.</li></ol><pre><strong>library</strong>(shiny)<br><br><em>## Define UI for your application</em></pre><pre>shinyUI(pageWithSidebar(<br><br>  <em># Application header</em><br>  headerPanel(&quot;My first Shiny app&quot;) ,<br>  <br>  <br> <em># Sidebar Panel to place the inputs and widgets and control the output </em><br>  sidebarPanel(<br>    <em>#slider input </em><br>    sliderInput(inputId = &quot;slider&quot;,<br>  label = &quot;Select a number&quot;,<br>  value = 10, min = 0, max = 100) ,<br><br>  <em>#a select input</em><br>  selectInput(inputId=&quot;selectid&quot;,label=&quot;Select the tyoe of plot&quot;,choices=c(&quot;ScatterPlot&quot;,&quot;Histogram&quot;)) <br>  ),<br>  <br>  <em>#main panel to display the outputs</em><br>  mainPanel(<br>    <br>    <em>#display the reactive plot build from the inputs </em><br>    plotOutput(&quot;plot&quot;) ,<br>    <br>    <em>#display some text</em><br>    verbatimTextOutput(&quot;text&quot;) # <strong>no , required to end the last <br>                                  # element of a section</strong></pre><pre>  )# end mainPanel<br> <br> ) #end PageWithSidebar<br> <br>) #end ShinyUI</pre><blockquote>Notice the , a comma after every section and every input and output element expect the last element in the section. Like there is no ,<strong> </strong>after verbatimTextOutput(&quot;text&quot;) as it is the last element of mainPanel(). It is important to add , otherwise the app would not run and show errors.</blockquote><h4>Combining all of this together, the steps are the following:</h4><ol><li>Begin each app with a simple shiny template as explained above.</li><li>Add elements as arguments to functions such as headerPanel(), sidebarPanel(), mainPanel().</li><li>Generate reactive inputs with an *Input() function.</li><li>Display reactive output with an *Output() function in the UI.</li><li>Assemble outputs from inputs in the server script.</li></ol><h3>Server.R</h3><p>Now you have to just assemble the inputs and outputs at the <strong>server-side</strong> script to add reactivity and the events which will occur. The server script will consist of all the actual R code and logic of the app. The basic task of a Shiny server script is to define the relationship between inputs and outputs. The server script does this by accessing inputs to perform computations and by assigning reactive expressions to output slots.</p><p>You have to start off with writing the server function. And this function takes parameters as input and output elements. server&lt;-function(input,output){ #Rcode }</p><ol><li>Accessing input using the input object and generating output by assigning to slots on the output object.</li><li>Save objects to display to output$plot at the server side from the ui, remember plotOutput(&quot;plot&quot;) was the output object defined in the <a href="http://127.0.0.1:5396/ShinyTutorial.Rmd#section-ui">UI</a>.</li><li>Build and generate objects to display using render*({#r-expressions}) function. There are different types of render functions based on the type of output you want to build. Such as renderPlot(),renderTable(),renderText(), renderUI() etc.The render*() the function is attached to the output object in the server script.</li><li>Access input values defined in the UI using input$slider. Here slider is the id given to the input object created in the UI as sliderInput(inputId=&quot;slider&quot;,....).The input value changes whenever a user changes the input.</li></ol><p>Below is a simple example to build a simple histogram.</p><pre>server&lt;-<strong>function</strong>(input,output) {<br>  <br>  output$plot&lt;- renderPlot({<br>    <br>    hist(rnorm(input$slider))<br>    <br>    }) <br>}</pre><p>This is how a simple server script would look like. Create reactivity by using Inputs to build rendered Outputs. Reactivity automatically occurs whenever you use an input value to render an output object.</p><p>Below is the demonstration of the above-explained shiny application.</p><h3>Example</h3><p>Let’s go step by step. Follow the steps given below each time you start making a shiny application. This will help you incrementally build an app and not mess up and avoid errors.</p><h4>Creating an empty shiny app</h4><ol><li>ui.R - create an empty layout and design the UI of your app.Divide the UI section into 3 different parts.</li><li>Header — Use headerPanel() to add the app title.</li><li>Sidebar Section — consists of the widgets and the input variables to control the output. Use sidebarPanel( *Inputs() ) to add input objects to the sidebar.</li><li>Main Section — display the outputs such as plots, text etc in this section. Use mainPanel( *Output() objects) to define output objects.</li></ol><pre><em> shinyUI( pageWithSidebar(</em><br><em>   </em><br><em>   # Application header</em><br><em>   </em><br><em>   #Side bar panel-inputs such as selectbox,widgets etc goes here</em><br><em>   </em><br><em>   #main panel-Reactive outputs are defined and displayed here</em><br><em>   </em><br><em>   </em><br><em>  ) #end pageWithSidebar<br>) #end shinyUI</em></pre><p>Now add header panels, sidebars and main panels using headerPanel(),sidebarPanel(),mainPanel() functions.</p><pre><em>shinyUI( pageWithSidebar(</em><br><em>   </em><br><em>   # Application header<br>   </em><strong><em>headerPanel(title=&quot;My first Shiny app&quot;)  ,</em></strong><em>   </em><br><em>   <br>#Side bar panel-inputs such as selectbox,widgets etc goes here </em></pre><pre><strong><em>   sidebarPanel(),  </em></strong><em> </em><br><em>   <br>#main panel-Reactive outputs are defined and displayed here</em><br><em>   <br>   </em><strong><em>mainPanel()</em></strong></pre><pre><em>   </em><br><em>  ) #end pageWithSidebar<br>) #end shinyUI</em></pre><blockquote>NOTE: notice no <strong><em>, </em></strong>after <strong><em>mainPanel()</em></strong>, as it is the last element of <em>pageWithSidebar()</em> object.</blockquote><p>Congratulations, now you have created and defined the basic structure of your application’s UI. You only need to define input and output objects inside the UI now.</p><p>All the functions used in the ui.R expect a default argument, without adding anything as an argument, the empty functions would give an error saying- <strong>Error:argument &quot;sidebarPanel&quot; is missing, with no default</strong>. So it is important that you comment out these functions initially if you have not added anything inside them.</p><blockquote>Don’t forget to add , after every section. Sometimes the errors arising due to missing , after a section can be really hard to debug.</blockquote><p>let’s build and place the input and output objects in the UI.</p><pre><em>## Define UI for your application</em></pre><pre>shinyUI( pageWithSidebar(<br><br>  <em># Application header</em><br>  headerPanel(&quot;My first Shiny app&quot;) ,<br>  <br>  <em># Side bar for inputs</em><br>  sidebarPanel(<br>    sliderInput(inputId = &quot;slider&quot;,<br>    label = &quot;Select a number&quot;,<br>    value = 10, min = 0, max = 100) ,<br>  <br><br><br>  <em>#a select input</em><br>  selectInput(inputId=&quot;selectid&quot;,label=&quot;Select the type of plot&quot;,choices=c(&quot;ScatterPlot&quot;,&quot;Histogram&quot;)) <br>  ), <br><br><br> <em># The main panel will contain the outputs</em><br>  mainPanel(<br>    <em>#display the reactive plot build from the inputs </em><br>    plotOutput(&quot;plot&quot;) ,<br>    <br>    <em>#display some text</em><br>    verbatimTextOutput(&quot;text&quot;)<br>    ) # end mainPanel<br>  <br>  ) #end pageWithSidebar</pre><pre>)#end shinyUI</pre><p>Now you have defined and placed the input and output objects in the UI of our application. You can design and structure the layout of your application in any way you want, it only requires your creativity and designing skills.</p><p>Do check out the documentation of the functions used above to build inputs and outputs using an example-?sliderInput to get to know more about these methods.</p><blockquote>Let’s summarize what you have done till now:</blockquote><ol><li>Begin each app with a simple shiny template as explained above.</li><li>Add elements as arguments to functions such as headerPanel(), sidebarPanel(), mainPanel().</li><li>Generate reactive inputs with an *Input() function.</li><li>Display reactive output with an *Output() function in the UI.</li><li>The only thing left now is to assemble outputs from inputs in the server script.</li></ol><p>Finally, let’s start with <strong>server.R</strong> and add some server logic to assemble inputs and output objects to add reactivity.</p><p>Follow these simple steps-</p><ol><li>Start off with defining the server function which takes parameters as input and output elements. server&lt;-function(input,output){ # R code goes here }</li><li>Save objects to display to output$plot at the server side from the ui, remember plotOutput(&quot;plot&quot;) was the output object defined in the UI.</li><li>Build and generate objects to display using render*({#r-expressions}) function. There are various other kinds of render*({}) functions in R to render different outputs.</li><li>Access the values stored in input objects defined in the UI using input$slider at server side. This is the key logic which adds reactivity. Here slider is the id given to the input object created in the UI using sliderInput(inputId=&quot;slider&quot;,....).</li></ol><pre><em>#server function takes input and output objects as arguments</em></pre><pre>server&lt;-<strong>function</strong>(input,output) {<br>  <br>  <em>#building the plots</em><br>  output$plot&lt;-renderPlot({<br>    <br>    <strong>if</strong>(input$selectid==&quot;ScatterPlot&quot;) {<br>       <em>#plotting the histogram</em><br>        plot(rnorm(input$slider),rnorm(input$slider),xlab=&quot;Numbers&quot;,ylab=&quot;Count&quot;,col=&quot;red&quot;,border=&quot;black&quot;) <br>      <br>    } <strong>else</strong> {<br>      <br>        <em>#plotting the histogram</em><br>        hist(rnorm(input$slider),col = &#39;darkgray&#39;, border = &#39;white&#39;)<br>      <br>    }<br>    <br>  }) #end output$plot<br>  <br>  <em>#building the textOutput</em><br>  output$text&lt;-renderPrint({<br>    <em>#summary statistics</em><br>     summary(rnorm(input$slider))<br>  <br>  }) #end output$text<br>  <br>}</pre><h3>Output</h3><p>When you run the above application you will get the following output-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aGP2E_8P6vQZ-7uSocKNGg.png" /><figcaption>The above app produces this output</figcaption></figure><h3>This was a simple example of how a shiny app would look and work.</h3><h3>Use Case</h3><h4>Kaggle data science survey App</h4><p>In this section, you will build a simple shiny application to analyze and visualize the Kaggle data science survey data. The dataset can be downloaded from <a href="https://www.kaggle.com/kaggle/kaggle-survey-2017">Kaggle Data science survey-2017</a>. Let’s start building the application.</p><p>You will use the Highcharter package for R to build plots. Highcharter uses the base function hchart() as the function to build plots. This function is similar to the qplot() from ggplot2 package. Read more about the package at <a href="http://jkunst.com/highcharter/hchart.html">Highcharter</a>.</p><p>One thing to notice is that when using highcharter with shiny, you need to use 2 functions-</p><ol><li>To define the output plot in UI- highchartOutput(output_id) instead of plotOutput() function.</li><li>To build the plot at the server-side use- renderHighchart({}) instead of renderPlot({}) function.</li></ol><p>Let’s begin.</p><h3>UI of the app</h3><pre>shinyUI( pageWithSidebar(<br><br>  <em># Application header</em><br>  headerPanel(&quot;Kaggle Data science survey data analysis&quot;) ,<br>  <br>  sidebarPanel(<br>  <em>#select country </em><br>  selectInput(inputId=&quot;country&quot;,label=&quot;Select the Country&quot;,choices=c(&quot;United States&quot;,&quot;India&quot;,&quot;United Kingdom&quot;,&quot;China&quot;,&quot;Germany&quot;,&quot;France&quot;),selected = NULL) <br>  ),<br><br>  <br>  <em>#the main section to display plots in tabs</em><br>  mainPanel (<br>  <br>        tabsetPanel(type = &quot;tabs&quot;,<br>                          <em>#tab1</em><br>                        tabPanel(&quot;Job Titles of participants&quot;, <br>                                   highchartOutput(&quot;plot1&quot;)<br>                                   <br>                                 ),<br>                    <br>                          <em>#tab2</em><br>                          tabPanel(&quot;Preferred tools&quot;,      <br>                               highchartOutput(&quot;plot2&quot;)<br>                                  <br>                                ),<br>                    <br>                          <em>#tab3</em><br>                          tabPanel(&quot;Preferred ML technique at Work&quot;,<br>                                highchartOutput(&quot;plot3&quot;)<br>                                   <br>                                )<br>                         )<em>#end tabsetpanel</em><br><br>                    )<em>#end mainpanel</em><br><br>                ) #end pageWithSidebar<br>              ) #end ShinyUI</pre><h3>Server script and logic of the application</h3><p>You will first start off by storing the dataset at the location where your application files are inside your application folder. Now you will read the dataset in server script and perform necessary data transformation and pre-processing as per the requirements of your application. One of the best packages in R to do data-wrangling and data-transformation is dplyr , and this is what you will use in this case study.</p><p>Dplyr is built around 5 functions. These functions make up the majority of the data manipulation you need to do. You might need to:</p><ol><li>select() certain variables of data.</li><li>filter() your data to select specific rows.</li><li>arrange() the rows of your data in a specific order.</li><li>mutate() your data frame to make new columns.</li><li>summarise() chunks of your data in some way.</li></ol><p>These functions are going to be used majorly inside the server script to do data manipulation.</p><p>In the server script shown below, you will first make a new filtered dataset countryJobs containing specific countries which you want to plot which are grouped by CurrentJobTitle variable and summarise them with total count using summarize(total=n()) function in R and then arrange them in descending order of total counts using arrange(desc(total)) function.</p><p>Then in a similar manner, you will create another data frame named countryToolDf in which you will group by a variable MLToolNextYearSelectwhich contains the tools used by the survey participants then filter it for specific countries passed-in filter() function and the rest is the same procedure. You summarise and arrange it in descending order.</p><p>You will then create another new data frame named countryMethodDf which will consist of the total count of most preferred machine learning technique used at workplace corresponding to each country by the survey participants.</p><p><strong><em>Now, these new data frames will be used inside the server function to filter data based on the selected user input.</em></strong></p><p>So the data transformation and pre-processing are done. Now you will start writing the server logic inside the server&lt;-function(input,output) { #logic } function.</p><pre><em>#requiring necessary R packages</em><br><br><strong>library</strong>(shiny)<br><strong>library</strong>(data.table) <em>#for faster data reading</em><br><strong>library</strong>(highcharter) <em>#visualization package based on javascript</em><br><strong>library</strong>(dplyr) <em>#for data-transformation and data-wrangling</em><br><br><br>SurveyDf&lt;-fread(&quot;Datasets/multipleChoiceResponses.csv&quot;)<br><br><em>#new filtered dataframe of country and corresponding count of job titles of survey participants</em><br>countryJobs&lt;-SurveyDf %&gt;% group_by(Country,CurrentJobTitleSelect) %&gt;%<br>        filter(Country %<strong>in</strong>% c(&quot;United States&quot;,&quot;India&quot;,&quot;Russia&quot;,&quot;United Kingdom&quot;,&quot;China&quot;,&quot;Brazil&quot;,&quot;Germany&quot;,&quot;France&quot;,&quot;Canada&quot;)) %&gt;%<br>        select(Country,CurrentJobTitleSelect) %&gt;%<br>        summarize(total=n()) %&gt;%<br>        arrange(desc(total))<br><br><em>#assigning NA value to missing values</em><br>countryJobs[1:2,]&lt;-NA<br><br><em>#excluding NA values </em><br>countryJobs&lt;-na.omit(countryJobs)<br><br><em>#making data frame for the tools used by countries participants</em><br>countryToolDf&lt;-SurveyDf %&gt;% group_by(Country,MLToolNextYearSelect) %&gt;%<br>        filter(Country %<strong>in</strong>% c(&quot;United States&quot;,&quot;India&quot;,&quot;United Kingdom&quot;,&quot;People &#39;s Republic of China&quot;,&quot;Germany&quot;,&quot;France&quot;)) %&gt;%<br>        select(Country,MLToolNextYearSelect) %&gt;%<br>        summarize(total=n()) %&gt;%<br>        arrange(desc(total))<br><br><em>#assigning missing values as NA</em><br>countryToolDf[c(1,2,10,11,12,13),]&lt;-NA<br><br><em>#excluding NA values</em><br>countryToolDf&lt;-na.omit(countryToolDf)<br><br><br><em>#making a new dataframe for countries and their most used ML techniques at Work place</em><br>countryMethodDf&lt;-SurveyDf %&gt;% group_by(Country,WorkAlgorithmsSelect) %&gt;%<br>        filter(Country %<strong>in</strong>% c(&quot;United States&quot;,&quot;India&quot;,&quot;United Kingdom&quot;,&quot;People &#39;s Republic of China&quot;,&quot;Germany&quot;,&quot;France&quot;)) %&gt;%<br>        select(Country,WorkAlgorithmsSelect) %&gt;%<br>        summarize(total=n()) %&gt;%<br>        top_n(20) %&gt;% <br>        arrange(desc(total))<br><br><em>#assigning missing values as NA</em><br>countryMethodDf[c(1,2,3,4,5,6),]&lt;-NA<br><br><em>#excluding NA values</em><br>countryMethodDf&lt;-na.omit(countryMethodDf)<br><br><br><br><em>#----------------------------------</em><br><br><em>#server function</em><br><br>server&lt;-<strong>function</strong>(input,output) {<br>  <br>  <em>#job titles of participants plot</em><br>  output$plot1&lt;-renderHighchart({<br>    <br>      <br>      <em>#making a data frame of the selected country to use in plotting</em><br>      df&lt;-countryJobs %&gt;% filter(Country==input$country)<br>      <br>        hchart(df,type=&quot;bar&quot;,name=&quot;Count&quot;, color=&quot;purple&quot;,hcaes( x=CurrentJobTitleSelect,y=total)) %&gt;%<br>        hc_title(text=&quot;Barplot of Jobs titles of participants&quot;,align=&quot;center&quot;) %&gt;%<br>        hc_exporting(enabled=TRUE) %&gt;%<br>        hc_add_theme(hc_theme_elementary())<br>        <br>    })<br>  <br>  <br>  <em>#most used Tool plot</em><br>  output$plot2&lt;-renderHighchart({<br>    <br>        df&lt;-countryToolDf %&gt;% filter(Country==input$country)<br>      <br>        hchart(df,type=&quot;bar&quot;,name=&quot;Count&quot;,color=&quot;red&quot;,hcaes( x=MLToolNextYearSelect,y=total)) %&gt;%<br>        hc_title(text=&quot;Barplot of Most used Tools and country&quot;,align=&quot;center&quot;) %&gt;%<br>        hc_exporting(enabled=TRUE) %&gt;%<br>        hc_add_theme(hc_theme_elementary())<br>    <br>  })<br>  <br>  <br>  <br>  <em>#most preferred ML technique plot</em><br>  output$plot3&lt;-renderHighchart({<br>    <br>        df&lt;-countryMethodDf %&gt;% filter(Country==input$country)<br>      <br>        hchart(df,type=&quot;bar&quot;,name=&quot;Count&quot;,color=&quot;orange&quot;,hcaes( x=WorkAlgorithmsSelect,y=total)) %&gt;%<br>        hc_title(text=&quot;Barplot of Most used ML technique and country&quot;,align=&quot;center&quot;) %&gt;%<br>        hc_exporting(enabled=TRUE) %&gt;%<br>        hc_add_theme(hc_theme_elementary())<br>    <br>  })<br>  <br>  <br>  <br>}</pre><h3>Output</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zUguD5m-D7FggVQwhouFJQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QbBWUSg7GyRMhzZZCa8Ucg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9dAeK3H6pUmTAznkAkCzXA.png" /></figure><p>Details about the various high charter package functions used above can be found using <strong>?highcharter</strong> help documentation.</p><p>The link to the above application’s Github Repository can be found here <strong>Github</strong>: <a href="https://github.com/anishsingh20/Kaggle-Data-science-survey-data-analysis-Using-Shiny">https://github.com/anishsingh20/Kaggle-Data-science-survey-data-analysis-Using-Shiny</a></p><p><strong>Deployed App: </strong><a href="https://anishwalia20.shinyapps.io/KaggleSurveyDataAnalysisDashboard/">https://anishwalia20.shinyapps.io/KaggleSurveyDataAnalysisDashboard/</a></p><p>Check the above link out to see this dashboard was built.</p><p>Hence this was a simple shiny application to visualize and analyze Kaggle survey data which shows you how to use highcharter package in R to build beautiful charts. You should definitely play around and find out more insights from the survey dataset using this simple app as a reference.</p><h3>Deploying your application</h3><p>Once you’ve built your shiny application, the next thing is to deploy it so that others can see your work and use the application. Rstudio provides a cloud platform which allows you to host your application on the web in minutes called <a href="http://www.shinyapps.io/">shinyapps.io</a>. It provides an amazing IaaS(infrastructure as a Service) which offers you the compute to host and run your shiny application for Free over the internet. You don’t need to own any kind of server. It has both free and paid options available.</p><p>You simply need to have an account on shinyapps.io, then it’s just the matter of minutes to upload your application over the platform and deploy it.Simply push-button publishing from RStudio IDE will do the needful. You need to follow this.</p><p>Below is an image which will guide you to deploy your application from your RStudioIDE.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zvW3za8XHW330yTTT4OOaQ.png" /></figure><p>So this is how you deploy your shiny application in a matter of minutes for free. Now the first 5 deployments are free on shinyapps.io, after that you will have to pay to deploy your application. More details regarding shinyapps.io can be found at this link <a href="http://docs.rstudio.com/shinyapps.io/">shinyapps.io documentation and user guide</a>.</p><h3>Shiny in production</h3><p>Other options to deploy your application in production with more compute power, active hours, more number of instances and most importantly in a more secure, scalable and fault-tolerant environment include the below options which are paid–</p><ol><li><strong>Deploy on-premises (open source)</strong>: <strong>Shiny Server</strong>-It helps you deploy your Shiny apps and interactive documents on-premises with open source Shiny Server, which offers features such as multiple apps on a single server and deployment of apps behind firewalls.</li><li><strong>Deploy on-premises (commercial)</strong>: <strong>Shiny Server Pro</strong>- Using this scheme you can deploy your apps on-premises which offers a controlled production environment with the security, control, and support that workgroups and enterprises need. Shiny Server Pro supports a variety of password authentication schemes and a centralized management console for all your apps. They have two commercially licensed platforms for deploying on-premises. They are both designed for maximum scalability and security, and they offer authentication and monitoring features.</li><li><strong>RStudio Connect</strong>-New publishing platform for the work your teams create in R. With RStudio Connect, you can share Shiny applications, R Markdown reports, dashboards, plots, and more in one convenient place with push-button publishing from the RStudio IDE. Features include scheduled execution of reports and flexible security policies to bring the power of data science to your entire enterprise.</li></ol><p>More details about the above details can be found at-<a href="https://shiny.rstudio.com/deploy/">https://shiny.rstudio.com/deploy/</a>.</p><h3>References</h3><p>Hence this was a simple tutorial on using the shiny package in R to develop web applications for data analytics and data science projects to effectively communicate the results and your key findings.</p><p>Below are some of the references and materials useful for learning more on developing a shiny web application.</p><ol><li>This course on <a href="https://www.datacamp.com/courses/building-web-applications-in-r-with-shiny?tap_a=5644-dce66f&amp;tap_s=210732-9d6bbf">Building Web applications in R using shiny</a> by DataCamp would be the best option to get started with learning Shiny.</li><li><a href="https://shiny.rstudio.com/tutorial/%5D">Shiny Tutorial</a>.</li><li><a href="https://datascienceplus.com/wp-content/uploads/2017/10/How-to-start-with-ShinyR.pdf">Shiny Cheatsheet</a></li><li><a href="https://shiny.rstudio.com/deploy/">Deploying a shiny app</a></li><li><a href="http://jkunst.com/highcharter/docs/index.html">Highcharter Documentation</a></li></ol><p>Hope you all liked the tutorial. Do Like and share Cheers!!.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f868f51a4122" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a COVID-19 Notifier in R]]></title>
            <link>https://anishsinghwalia.medium.com/creating-a-covid-19-notifier-in-r-d10bde45da3e?source=rss-3bfe9db19781------2</link>
            <guid isPermaLink="false">https://medium.com/p/d10bde45da3e</guid>
            <category><![CDATA[rstudio]]></category>
            <category><![CDATA[application-development]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Tattooed Geek]]></dc:creator>
            <pubDate>Sat, 09 May 2020 04:58:05 GMT</pubDate>
            <atom:updated>2023-12-14T11:43:31.710Z</atom:updated>
            <content:encoded><![CDATA[<h4>In this tutorial, we are going to create a desktop Notifier in R using the <a href="https://github.com/gaborcsardi/notifier">Notifier</a> package.</h4><p>I will divide the post into 3 parts to properly explain to you the basics and build up complex Notifiers:</p><ol><li><strong>A basic desktop notifier using notifier() in R.</strong></li><li><strong>Creating a desktop notifier to send notifications after regular intervals.</strong></li><li><strong>Creating a desktop notifier to send COVID-19 updates after every 1 hour.</strong></li></ol><p>It is really simple and straight forward. You can use this to create reminders, applications and motivating quotes for yourself which you can tweak accordingly to send you reminders at the regular time intervals. In this post, I will also explain to you about sending notification at a certain time or at regular time intervals.</p><p>Link to a few of the applications/dashboards I have developed in R.</p><ol><li><a href="https://anishsingh.shinyapps.io/Corona-Virus-Spread-WebApp/">Analysing Corona Virus spread </a><a href="https://anishsingh.shinyapps.io/Corona-Virus-Spread-WebApp/">Worldwide</a> <a href="https://anishsingh.shinyapps.io/Corona-Virus-Spread-WebApp/">in R</a></li><li><a href="https://anishsingh.shinyapps.io/COVID-19IndiaTracker/">COVID-19 India Tracker in R</a></li><li><a href="https://anishwalia20.shinyapps.io/IndianEconomyAnalysis-App/">Analysis of Indian Economy using R</a></li><li><a href="https://anishwalia20.shinyapps.io/KaggleSurveyDataAnalysisDashboard/">kaggle Data Science Survey Data Analysis Dashboard</a></li></ol><p>So go ahead and check out these dashboards/web-apps made in R. Use it as a reference to create your own projects. The link to the code and GitHub repositories are in the App.</p><blockquote><strong>Bonus Tip</strong>: One great tool I recently started using for writing and tasks such as plagiarism checker, grammar checker, Co-writer, paraphraser, summariser, and translator is <a href="https://try.quillbot.com/mimzuo3pn068">QuillBot</a> .</blockquote><blockquote>I wanted to try something similar and cheaper than <strong>Grammarly</strong>.</blockquote><blockquote>I took up its <strong>yearly premium for around $2/month (45% off) during the Year-end sale using coupon code — (HOLIDAY45),</strong> valid till <strong>December end</strong>. The price was literally dirt cheap compared to other writing tools I have used in the past.</blockquote><blockquote>Personally, it’s UI and UX is very simple and easy to use. So I just wanted to share this awesome, productive tool with you all. Do check it out and use it in your day-to-day writing tasks.</blockquote><blockquote><a href="https://try.quillbot.com/mimzuo3pn068"><strong>https://try.quillbot.com/</strong></a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tntj4it_zNgcEqhg6jwUdA.png" /><figcaption>Best Productivity Writing tool for this month</figcaption></figure><blockquote><strong>SIDE NOTE:</strong></blockquote><blockquote><em>I always make sure I share my knowledge and resources which helped me. So I will be adding a few links to my favourite courses below related to this post and which can help you experiment with this post on your own</em>. My focus has always been to learn things in a practical manner, that is why <a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp</em></strong></a> has always been my go-to place for learning from the very start. It’s just that in this manner you can actually learn faster and better. <a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp</em></strong></a> also has an amazing library of Projects. They started this new service recently and it’s really amazing. <a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp projects</em></strong></a><strong><em> </em></strong>is another amazing thing which I keep doing from time to time. Make sure to give it a try. Just once try them and then you will understand why I love <a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp</em></strong></a> so much.</blockquote><p><strong>DataCamp will be offering a first-month subscription for only $1 from today till 3rd Dec, 8 PM ET. Also, they are having a black Friday sale where Yearly subscriptions have 75 % off. This is the first time they have ever run a sale like this. So do make sure to go check out </strong><a href="https://datacamp.pxf.io/x9NMDy"><strong>DataCamp</strong></a><strong>. Also, </strong><a href="https://datacamp.pxf.io/x9N7JA"><strong><em>DATCAMP CHEATSHEETS</em></strong></a><strong> is a pretty handy and very useful tool that I use from time to time.</strong></p><blockquote><em>Firstly, If you are not familiar with the basics of the R language, I urge readers to go complete this amazing course by </em><a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp</em></strong></a><em> on </em><a href="https://datacamp.pxf.io/kjoBdx"><strong><em>R Programming</em></strong></a><em>, and this is the course from where I started my Data science journey using R years ago. Trust me with this, these courses are worth your time and money or if you want to learn the complete data science fundamentals from scratch using R which includes Statistics and probability, Data viz, exploring the data, data modelling and machine learning and other important concepts, I strongly recommend this foundation course </em><a href="https://datacamp.pxf.io/QO6kV6"><strong><em>Data Science with R</em></strong></a><em> .</em></blockquote><blockquote><a href="https://datacamp.pxf.io/kjoBdx"><strong>Introduction to writing functions in R</strong></a><strong>.</strong></blockquote><blockquote><a href="https://datacamp.pxf.io/oeyEMe"><strong>Introduction to machine learning in R</strong></a><strong>.</strong></blockquote><blockquote><a href="https://datacamp.pxf.io/za2LMx"><strong>Writing efficient code in R</strong></a><strong>.</strong></blockquote><blockquote><a href="https://datacamp.pxf.io/7mJjaO"><strong>Building web apps in R using Shiny.</strong></a></blockquote><blockquote><em>So this would literally be the best time to grab some yearly subscriptions(</em><strong><em>which I have</em></strong><em>) which basically has unlimited access to all the courses and other things on </em><a href="https://datacamp.pxf.io/x9NMDy"><strong><em>DataCamp</em></strong></a><em> and make fruitful use of your time sitting at home during this Pandemic. So go for it folks and Happy learning, make the best use of this quarantine time and come out of this pandemic stronger and more skilled.</em></blockquote><h4>How do Notifier works?</h4><ol><li><strong>MAC OS</strong>: It uses MAC’s terminal-Notifier tool(<a href="https://github.com/julienXX/terminal-notifier">https://github.com/julienXX/terminal-notifier</a>).</li><li><strong>Linux based and BSD based systems</strong>: It uses Notify-send command-line tool is used. This requires the “<strong>libnotify</strong>“-bin package on Ubuntu/Debian and similar systems or the <strong>libnotify</strong> package on RedHat/CentOS/Fedora and similar systems.</li><li><strong>Windows</strong>: Windows versions, notifier uses the toaster tool, see <a href="https://github.com/nels-o/toaster">https://github.com/nels-o/toaster</a>.</li></ol><h4>Usage of Notifier</h4><p><strong>“notifier”</strong> uses one basic function to create a notification or as the function says, “notify“. The function notify() takes the below three arguments:</p><ol><li>title — Message title.</li><li>msg — Actual Notification Message.</li><li>image —add Image, along with the message — optional.</li></ol><p>Below is a basic example of using <strong>notify() </strong>function.</p><pre><strong>#installing the Notifier package</strong></pre><pre>devtools::install_github(“gaborcsardi/notifier”)</pre><pre>library(notifier)</pre><pre><strong>notify(</strong></pre><pre><strong>title = “This is Anish”,</strong></pre><pre><strong>msg= c(“Hi”,”this is my first desktop notifier”)</strong></pre><pre><strong>)</strong></pre><pre># For more details about the function you can use:<br># ?notify</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*svgQHMQWw212fQeU7NepMA.png" /><figcaption>Output</figcaption></figure><p>And this is it. Just one function <strong>notify(). </strong>You will notice notifications popping up on your desktop.</p><h4>Making a notifier which will send notifications after regular Intervals:</h4><p>There are multiple use cases for the desktop notifications for example you want to get notified every time the script gets completed i.e after a specific time interval.</p><pre># <strong>making a function to send notifications every 1 minute</strong></pre><pre>#&#39;randquotes&#39; for generating random quotes<br><strong>library(randquotes)</strong></pre><pre>repeat {<br>  <br>  # getting the current time<br>  startTime &lt;- Sys.time() <br>  <br>  <br>  notify(<br>    <br>    title = &quot;Though of the day:&quot;,<br>    msg = c(randquote_simple())<br>)</pre><pre>#the logic is simple (Start_time + time_interval in seconds - #current_sys_time) will give us the sleep time i.e interval after #which notifications is again sent  </pre><pre>sleepTime &lt;- startTime + 60 - Sys.time()<br>  if (sleepTime &gt; 0)<br>  Sys.sleep(sleepTime)</pre><pre>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*xmpYVjO2Ou3jdjU3EcavkQ.png" /><figcaption>Output</figcaption></figure><p>So this is it. Another simple implementation and no rocket science.</p><h4>Making a notifier which will send notifications about COVID-19 cases for India every hour</h4><p>Let’s first get the data from the data source. Here you can scrape data from any source in R. The process is fairly simple.<strong> </strong><a href="http://r-bloggers.com/using-rvest-to-scrape-an-html-table/"><strong>This</strong></a> is a really good post on scraping data in R using a really good package <a href="http://r-bloggers.com/using-rvest-to-scrape-an-html-table/"><strong>rvest</strong></a><strong>. </strong>You can follow this article to scrape web data in R.</p><p>Here we are also going to use another amazing R package “<a href="https://www.r-project.org/nosvn/pandoc/beepr.html"><strong>beepr</strong></a>” to add notification sound tones to make it better. BeepR simply uses a function “<strong>beep()</strong>”. We are going to use “<strong>RCurl</strong>” package to read data in R.</p><pre><strong>require(RCurl)</strong></pre><pre>myfile1 &lt;-getURL(&#39;https://api.covid19india.org/csv/latest/state_wise.csv&#39;, <br>                    ssl.verifyhost=FALSE, ssl.verifypeer=FALSE)<br>  <br>  #dataset with summarised COVID-19 case counts of Indian states<br>  StateCOVID_19 &lt;- read.csv(textConnection(myfile1), header=T)</pre><pre><br> msg1=paste(&quot;Confirmed Cases: &quot;,Confirmed,&quot;, Recovered:&quot;,Recovered)<br> msg2=paste(&quot;Deaths: &quot;,Deaths, &quot;, Active:\n &quot;,Active)<br> <br><br> <br>repeat {<br>  <br>  # getting the current time<br>  <strong>startTime &lt;- Sys.time() </strong><br>  <br>  <br>notify(<br>    <br>    title = &quot;COVID-19 India&#39;s Summary&quot;,<br>    msg = c(msg1,msg2)<br>)<br>  <br>#adding the beep notification sound<br><strong>beep(&quot;wilhelm&quot;)</strong><br><br>  #send notification after every <strong>hour(3600 seconds)</strong><br>  sleepTime &lt;- startTime + <strong>3600</strong> - Sys.time()<br>  if (sleepTime &gt; 0)<br>  Sys.sleep(sleepTime)<br><br>  <br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/334/1*W5yfnOp9ylPWVJECUViR9A.png" /><figcaption><strong>The output of the above function</strong></figcaption></figure><p>And that is it. Go ahead and create your own notifier in R.</p><p>I hope you guys liked the post. Make sure to like it and share it with your colleagues. Use this post to make something interesting and experiment with it.</p><p>You can reach out to me on the following social profiles below for any doubts or queries and follow me there as well.</p><p>Stay safe and stay indoors and make sure to make the best use of your time during this lockdown.</p><ol><li><a href="https://github.com/anishsingh20">Github</a></li><li><a href="https://www.linkedin.com/in/anish-singh-walia-924529103/">LinkedIn</a></li><li><a href="https://rpubs.com/anish20">RPubs</a></li><li><a href="https://datascienceplus.com/author/anish-singh-walia/">DataScience+</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d10bde45da3e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>