<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lalit Soni - Cloud Engineering | MLOps | DevOps on Medium]]></title>
        <description><![CDATA[Stories by Lalit Soni - Cloud Engineering | MLOps | DevOps on Medium]]></description>
        <link>https://medium.com/@lalit-soni?source=rss-50016ee8d45d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*bS8NqgAdgsu9hXeziuyVww.jpeg</url>
            <title>Stories by Lalit Soni - Cloud Engineering | MLOps | DevOps on Medium</title>
            <link>https://medium.com/@lalit-soni?source=rss-50016ee8d45d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:10:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lalit-soni/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Meta’s most advanced and powerful LLAMA 3.1 AI opensource model released]]></title>
            <link>https://faun.pub/metas-most-advanced-and-powerful-llama-3-1-ai-opensource-model-released-e6071a171003?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6071a171003</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[llama-3-1]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Thu, 25 Jul 2024 05:45:24 GMT</pubDate>
            <atom:updated>2024-07-30T08:14:55.463Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="lalitsoni.com blog published for META AI Opensource LLAMA 3.1 model" src="https://cdn-images-1.medium.com/max/738/1*a3KoiA4mVNgDOkfYqzyGLg.png" /><figcaption>credits: shutterstock.com / META AI</figcaption></figure><p>On 23&#39;rd July 2024, Mark Zuckerberg (CEO of Facebook/Meta) released a video on Instagram by self recording with the information about their new advancement in AI by Meta. Zuckerberg announced about Meta’s new opensource AI model named LLAMA 3.1 which is most powerful and capable to date.</p><p>LLAMA 3.1 have three category of model on the basis of Parameters 405B, 70B and 8B.</p><p>Reason why this opensource by Meta makes the tech industry exciting because it gives fair competition to its rival other private company paid models like OpenAI’s GPT 4.0 and Anthropic’s Claude 3.5 Sonnet, etc.</p><p><strong>1. LLAMA 3.1 405B </strong>:- According to Meta, It is world’s largest publicly available LLM (Large Language Model). The model sets a new standard for AI and is ideal for enterprise-level applications and research and development (R&amp;D). This model excels at general knowledge, long-form text generation, multilingual translation, machine translation, coding, math, tool use, enhanced contextual understanding</p><p><strong>2. LLAMA 3.1 70B</strong> :- This is perfect ideal model for content creation, conversational AI, language understanding, R&amp;D, and enterprise applications. The model excels at text summarization and accuracy, text classification, sentiment analysis and nuance reasoning, language modeling, dialogue systems, code generation, and following instructions.</p><p><strong>3. Llama 3.1 8B :- </strong>It<strong> </strong>is best suited for limited computational power and resources. The model excels at text summarization, text classification, sentiment analysis, and language translation requiring low-latency inferencing.</p><figure><img alt="meta ai opensource LLAMA 3.1 most advanced AI model — lalitsoni.com" src="https://cdn-images-1.medium.com/max/290/1*iw_JdWK5Tw6LA4zrkFzvYQ.jpeg" /><figcaption>Meta AI</figcaption></figure><h3><strong>Technical Innovation in LLAMA 3.1 :-</strong></h3><p>LLaMA 3.1 integrates several groundbreaking advancements:</p><h3><strong>Extended Context Length:</strong></h3><p>LLaMA 3.1 supports a context length of up to 128K, allowing for a more comprehensive understanding and generation of long-form content.</p><h3><strong>Multilingual Capabilities:</strong></h3><p>The model supports eight languages, enhancing its usability in diverse linguistic contexts and applications.</p><h3><strong>Enhanced Architecture:</strong></h3><p>Utilizing a standard decoder-only transformer model, LLaMA 3.1 incorporates minor adaptations to maximize training stability and efficiency. The model was trained on over 15 trillion tokens using more than 16,000 H100 GPUs, a significant technical feat.</p><h3><strong>Synthetic Data Generation and Model Distillation:</strong></h3><p>This model introduces unprecedented capabilities in synthetic data generation, enabling the training of smaller models with high-quality synthetic data. Model distillation at this scale is now possible, providing a new dimension for AI development.</p><h3><strong>Quantization:</strong></h3><p>The models are quantized from 16-bit (BF16) to 8-bit (FP8) numerics, reducing computational requirements and enabling efficient deployment on single server nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LUHfY2riW3EEIXHm" /><figcaption>Photo by <a href="https://unsplash.com/@cgower?utm_source=medium&amp;utm_medium=referral">Christopher Gower</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Meta AI’s Ecosystem and Freedom to Contribute</h3><p>Meta is visionary company in order to upgrade technology and making it accessible to everyone. LLAMA 3.1 is a frontier in opensource AI community while other rivals are getting paid to use GPT 4.o, etc.</p><p>From the day one, Meta has partnered with many companies including AWS, Nvidia, Databricks, Accenture, GCP, Azure, etc to provide robust support for LLAMA 3.1.</p><p>Open-sourcing the model makes every developer to access the power of AI model (LLAMA 3.1) and customize it as per their interest.</p><p>Meta knows the significance of Resposible AI. Meta committed and already following the nature of Responsible AI as they have not used META product’s data to train this model. They have used Public data and claimed that less to no PII Data is included while training the model.</p><p>If you want daily updates in advancement of Technology please subscribe to this channel. You can connect me virtually on <a href="http://lalitsoni.com/">lalitsoni.com</a></p><p>Thank you for being an awesome reader <strong>😃</strong></p><p>Please 👏 👏 the article for giving Positive Vibes to me <strong>😃.</strong></p><p>Check out my other blogs also.</p><ul><li><a href="https://lalit-soni.medium.com/role-of-ai-in-devops-aiops-mlops-bbfb5b74c9ef">Role of AI in DevOps — AIOps MLOps</a></li><li><a href="https://lalit-soni.medium.com/meta-horizon-os-next-gen-mixed-reality-ecosystem-763f43ac717d">Meta Horizon OS — Next Gen Mixed Reality Ecosystem</a></li><li><a href="https://faun.pub/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c">What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?</a></li><li><a href="https://lalit-soni.medium.com/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">What is ELK Stack? Elasticsearch, Logstash and Kibana</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*fV1WLObwrTh8ucLE.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6071a171003" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/metas-most-advanced-and-powerful-llama-3-1-ai-opensource-model-released-e6071a171003">Meta’s most advanced and powerful LLAMA 3.1 AI opensource model released</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Meta Horizon OS — Next Gen Mixed Reality Ecosystem]]></title>
            <link>https://lalit-soni.medium.com/meta-horizon-os-next-gen-mixed-reality-ecosystem-763f43ac717d?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/763f43ac717d</guid>
            <category><![CDATA[meta]]></category>
            <category><![CDATA[meta-horizon-os]]></category>
            <category><![CDATA[cloudops]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mixed-reality]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Wed, 24 Apr 2024 10:09:32 GMT</pubDate>
            <atom:updated>2024-04-24T10:09:32.780Z</atom:updated>
            <content:encoded><![CDATA[<h3>Meta Horizon OS — Next Gen Mixed Reality Ecosystem</h3><h3>Introduction</h3><p>Just Now got an amazing update from the Founder and CEO of meta i.e Mr. Mark Zuckerberg about their new Next Generation Mixed Reality Platform named<a href="https://en.wikipedia.org/wiki/Meta_Horizon_OS"> META HORIZON OS</a> which will revolutionize the market and sets new standard of how we utilize the power of computing.</p><figure><img alt="meta horizon os credit: meta" src="https://cdn-images-1.medium.com/max/300/1*EG6CL3QtTx3jWUWvHy_LRQ.jpeg" /><figcaption>credit: meta platforms inc.</figcaption></figure><h3>Meta Horizon OS</h3><p>In the era of AI, Meta is working on this Mixed Reality platform from a Decade and Now they have decided to work on it using Open Model. Mr. Mark explained there are 2 types of model usually gets success for computing devices in market 1. Closed Model 2. Open Model. Closed model is usually and Apple devices where everything is locked and controlled by Apple even what world developers will build on their Platform that is also controlled. But In Open Model, Anyone can bring the creativity on table and build on top of their existing Platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*jMRwKP0yJitzmISoRjFjzA.jpeg" /><figcaption>mixed reality platform credit: Meta Platform Inc.</figcaption></figure><p>For Meta Horizon OS they opted for Open Model where everyone is allowed to work and develop on top of their existing product.</p><p>Let me take you in more depth about Meta Horizon OS. It is developed by Meta Platforms Inc. Main goal for the Meta Horizon OS is to target Extended Reality Headset and Meta Quest Headset. Meta started working on it as Embedded OS for Oculus VR devices and now They developed Extended Reality OS for the Meta Quest line of deviced by Meta Platforms Inc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V1YFVNY-vzyHEGiEF5zepQ.jpeg" /><figcaption>mark on Instagram</figcaption></figure><p>Meta is pushing the Extended reality and Virtual Reality to work upon many advance real life use cases which will change the use of computing power. An immersive experience of Virtual Spaces will bring the revolution to connect people in more realistic experience. Some usecase of Mixed Reality is Exercise tracking, Real time Video conferences will physical appearance in virtual spaces, multiple Big Screen can be displayed on virtual spaces instead of occupying room.</p><p>There will be headsets focused purely on watching TV and movies on virtual screens, with the emphasis on high-end OLED displays. There will also be headsets that are designed to be as light as possible at the expense of performance for productivity and exercise uses. And there will be gaming-oriented ones.</p><p>Meta Platform Inc. announced three partners to start. Asus will produce a gaming headset under its Republic of Gamers (ROG) brand, Lenovo will make general purpose headsets with an emphasize on “productivity, learning, and entertainment,” and Xbox and Meta will team up to deliver a special edition of the Meta Quest that will come bundled with an Xbox controller and Xbox Cloud Gaming and Game Pass.</p><p>Users running Horizon OS devices from different manufacturers will be able to stay connected in the operating system’s social layer of “identities, avatars, social graphs, and friend groups” and will be able to enjoy shared virtual spaces together across devices.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=763f43ac717d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Role of AI in DevOps — AIOps MLOps]]></title>
            <link>https://lalit-soni.medium.com/role-of-ai-in-devops-aiops-mlops-bbfb5b74c9ef?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbfb5b74c9ef</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Wed, 27 Mar 2024 09:24:36 GMT</pubDate>
            <atom:updated>2024-03-27T09:24:36.281Z</atom:updated>
            <content:encoded><![CDATA[<h3>Role of AI in DevOps — AIOps MLOps</h3><h3>Introduction</h3><p>DevOps is growing and demanding in recent times to automate the Software Workflows and Deployment. AI is insanely progressing to be incorporated in every domain i.e Marketing, Entertainment, Banking and Financial Services, Insurance, Retail.</p><p>AI (Artificial Intelligence) already open up the new arenas to have a exponential speed to innovate and code faster for the businesses and Technology Companies. AI is growing day by day with huge Research and Development on it.</p><p>All product based Technology companies are heading toward to harness the power of AI, Machine Learning. Everyone is trying to incorporate the AI power in their Products like Atlassian recently launched their BETA version on Atlassian Intelligence.</p><p>To Improve the performance and efficiency of the product, AI is best way to work upon. NLP, CV (Computer Vision), LLMs, etc are great way to research and use currently.</p><h3>AI + DevOps = AIOps</h3><p>New term is coined named AIOps which denoted basically automation of the AI Operations. In order to deploy and get the real time result from the model require proper deployment strategy which can be easily Scalable so that ML models can be use by mass users. Deployment is always the challenge when we consider to serve to mass users. Compute power is still a factor for the mid size companies to optimize upon for less expensive ML solution serving.</p><p>DevOps can be more faster and quicker using AI. AI will act as a catalyst for performing DevOps activities. AI is a big revolution happen for this generation. Many DevOps tools have already incorporated AI in their products for optimizing and as a added Features.</p><p>Delivering and Deploying the Software products faster DevOps principles are still one of the best way to follow for the mid-size and large-companies. Security will be always concern for the companies while adapting the new systems and solution by AI. AI can help us to make our systems more secure but still securing an AI solution will be major concern while adopting any AI solution in production systems.</p><figure><img alt="AI in DevOps, AIOps, MLOps, LalitSoni.com, Lalitsoni, CloudOps, Cloud Engineer" src="https://cdn-images-1.medium.com/max/800/1*yCh7Xj4d4Lv12MkqexyvgQ.png" /></figure><h3>AI can improve DevOps several ways :</h3><h4>1. Automated Testing:</h4><ul><li>AI-powered testing tools can analyze code changes and automatically generate test cases, speeding up the testing process and increasing test coverage. AI algorithms can also detect patterns in test results to identify areas for improvement.</li></ul><h4>2. Predictive Analytics:</h4><p>AI can analyze historical data from development and operations processes to predict potential issues before they occur. For example, AI can forecast resource usage trends to optimize infrastructure provisioning or predict software bugs based on past code changes.</p><h4>3. Continuous Integration/Continuous Deployment (CI/CD):</h4><p>AI can optimize CI/CD pipelines by automatically identifying bottlenecks, suggesting optimizations, and predicting the impact of code changes on system performance. This helps streamline the deployment process and improve software delivery speed and reliability.</p><h4>4. Performance Monitoring and Optimization:</h4><p>AI-powered monitoring tools can analyze system metrics in real-time to detect anomalies and performance degradation. By automatically identifying and addressing performance issues, AI improves system reliability and user experience.</p><h4>5. Security and Compliance:</h4><p>AI can enhance security and compliance in DevOps by automatically detecting security vulnerabilities, identifying suspicious behavior, and enforcing compliance policies. This helps organizations mitigate security risks and maintain regulatory compliance throughout the development and deployment lifecycle.</p><h4>6. Infrastructure Management:</h4><p>AI can optimize resource allocation and automate infrastructure provisioning and scaling based on workload demands. By dynamically adjusting resources in response to changing conditions, AI improves resource utilization and reduces operational costs.</p><p>Overall, AI brings automation, intelligence, and predictive capabilities to DevOps processes, enabling organizations to streamline their development and operations workflows, increase productivity, and deliver higher-quality software faster and more efficiently.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbfb5b74c9ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[All About CRED’s Tech Stack -Indian Fintech Startup]]></title>
            <link>https://lalit-soni.medium.com/all-about-creds-tech-stack-indian-fintech-startup-d41c8c2a9994?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/d41c8c2a9994</guid>
            <category><![CDATA[credit-cards]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[fintech]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 01:10:53 GMT</pubDate>
            <atom:updated>2021-10-18T01:10:53.181Z</atom:updated>
            <content:encoded><![CDATA[<p>In this Blog, I will showcase the whole Technology Stack <a href="https://cred.club/"><strong>CRED</strong></a> is using to make their Service Accessible to the Customers.</p><h3>All about CRED</h3><p>First of all we will discuss about <a href="https://cred.club/"><strong>CRED</strong></a>, so <a href="https://cred.club/"><strong>CRED</strong></a> is a fintech startup of Bengaluru, India founded by Entrepreneur KUNAL SHAH. KUNAL already had experience to run the fintech startup in Indian Markets earlier i.e FREECHARGE. Kunal also is a Investor, he heavily invested in lots of Tech Startups. So lets talk about CRED.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*1d8IjdEsiI9ZjUYb" /><figcaption>www.cred.club</figcaption></figure><p>CRED’s main feature is allowing users to make credit card payments through its app for which they get rewarded. Later, CRED added features to allow users to make house rent payments and introduced short-term credit lines.</p><h3>CRED’s Engineering Team</h3><p>Recently, CRED’s engineering team came to one of the online event to discuss about their working culture in the company. They also discussed about Technology Stack they are using for their products. CRED have more than <strong>10 Inhouse Products </strong>to deal with their Customers.</p><p>Also CRED’s Engineering team designed their Architecture in such a way that it can be scalable with Availabilityand Reliability, no stress to deal with Infrastructure in real time. Also They are using DevOps for different purpose like code quality checks, testing, CI/CD and fast delivery of the product. Team is very much concerned for long term so they are using different techniques i.e <strong>Chaos Engineering</strong> to make their Infrastructure fault tolerant, Availability and reliable.</p><p>So Let me show you the whole Tech Stack they are using, I know you are excited <strong>😃 😃 😃 😃</strong></p><h3>For Application and Data:-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/0*_C-EbSFW8SbcVDMp.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*Ig26KzpnZPk39iYz.png" /></figure><h3>Utilities :-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/0*qxml3yoLvg05_Py9.png" /></figure><h3>DevOps Tools :-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/0*gwy9jWGCwtZgdJhX.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/0*AZr5LAcVHP2sOdbL.png" /></figure><h3>Business Tools :-</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/0*Zft-d4DbdO1dhAzl.png" /></figure><h3>Conclusion</h3><p>A good number of technologies CRED’s team is using. CRED’s users is increasing daily and also it is Fintech Company so they need to trace every Transaction or activity happening inside product.</p><p>I hope this article will be helpful to you. It will expand your view toward an Engineering system.</p><p>Please 👏 👏👏 👏 the Article to send some Positive Vibes.</p><p>Also Check out my other Posts as i always try to bring something new on the Table.</p><ul><li><a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">What is ELK Stack? Elasticsearch, Logstash and Kibana</a></li><li><a href="https://medium.com/@lalit.soni/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c">What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?</a></li><li><a href="https://medium.com/@lalit.soni/how-to-analyse-jenkins-build-logs-with-elasticsearch-logstash-and-kibana-elk-stack-ad372429858d">How to Analyse Jenkins Build Logs with Elasticsearch, Logstash and Kibana (ELK Stack) ?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d41c8c2a9994" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Analyse Jenkins Build Logs with Elasticsearch, Logstash and Kibana (ELK Stack) ?]]></title>
            <link>https://lalit-soni.medium.com/how-to-analyse-jenkins-build-logs-with-elasticsearch-logstash-and-kibana-elk-stack-ad372429858d?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad372429858d</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[live-streaming]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 01:05:59 GMT</pubDate>
            <atom:updated>2021-10-18T01:05:59.700Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to Analyse Jenkins Build Logs with Elasticsearch, Logstash and Kibana (ELK Stack) ?</h3><blockquote><em>Now a days Jenkins has been used in many companies to automate their build tasks and Elasticsearch to store documented data to visualization the data via Kibana.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/869/0*qRQNw5nTg1ArMqE7.png" /><figcaption>Jenkins Logs to ELK Stack</figcaption></figure><p>Hello there, In my last article, I have explained about ELK Stack and Why we monitor the Infrastructure Logs, Application Logs, etc. If you are newbie to ELK Stack, <a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">you can check out my article on ELK Stack by clicking here.</a></p><h4><a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c?source=user_profile---------1----------------------------">What is ELK Stack ? Elasticsearch, Logstash and Kibana</a></h4><p>In this blogpost, I will explain how to Analyse Jenkins Build Logs on your Kibana Dashboard. So as we already know Jenkins is an automation server (tool). Whenever it comes to monitoring the system, “Analysis of Jenkins Logs” is important because lots of companies are using it as an Automation tool.</p><p>That’s why in this blog post, we will learn how we can fetch the Jenkins Build Logs to our ELK Server and visualize it on Kibana Dashboard.</p><p>Prerequisite to perform this tutorial:-</p><h4>Jenkins server and ELK server are required to perform this Hands-on tutorial.<a href="https://medium.com/@lalit.soni/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c?source=user_profile---------0----------------------------"> To learn about Jenkins and Setup Jenkins Server you can check out my blog post on it.</a></h4><p><a href="https://medium.com/@lalit.soni/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c">What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?</a></p><p><strong>STEP 1 :-</strong></p><ol><li>Setup Jenkins are ELK server on VMs(Virtual Machine are preferable) by doing Installation 1. Jenkins and 2. ELK</li></ol><p><strong>STEP 2 :-</strong></p><ol><li>Install<strong> Logstash</strong> Plugin in Jenkins</li><li>Login to Jenkins &gt; Manage Jenkins &gt; Plugin Manager &gt; “Click Available Tab” &gt; search “Logstash” in the filter box, then install it.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*LZ1EnAG_dyumkkfO" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/0*UBqZeIecf9paKO0G.png" /><figcaption>Install Logstash Plugin in Jenkins</figcaption></figure><p><strong>STEP 3 :-</strong></p><p>Create Index in Elasticsearch, Jenkins sends the build job logs to this Elasticsearch Index. Use this command :<br> <strong>curl -X PUT “Elasticsearch_IP:9200/Index_name?pretty”</strong><br> eg :<br> <strong>curl -X PUT “192.168.56.136:9200/logstash?pretty”</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*N2WL0hPGZ-7X44rY" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*CfXbVGKgjG5vNImz.png" /><figcaption>New Index in Elasticsearch</figcaption></figure><p><strong>STEP 4 :-</strong></p><p>Configure Logstash plugin in Jenkins,<br> <br> Login to Jenkins &gt; Manage Jenkins &gt; Configure System &gt; find “Logstash” and do the following,<br> <br> 1. enable “Enable sending logs to an Indexer”<br> 2. Indexer Type: Elastic Search<br> 3. URI: <a href="http://elastisearch_ip:9200/Index_name/type">http://Elastisearch_IP:9200/Index_name/type</a><br> <br> eg : <a href="http://192.168.56.136:9200/logstash/jenkins">http://192.168.56.136:9200/logstash/jenkins</a><br> <br> If you have a username and password, fill it and save the configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*kH0CmTtF94OKJ2fu" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*iK_teDAlzd8E1YQK.PNG" /><figcaption>Configure Logstash Plugin</figcaption></figure><p><strong>STEP 5 :-</strong><br> <br> Create and Configure Jenkins Jobs to send build jobs to Elasticsearch,<br> <br>1. Create a Job,<br> <br> Login to Jenkins &gt; New Item &gt; Enter Job name &gt; Choose the project type (I used Freestyle project)<br> <br> 2. Go to Build Environment part, click “Add build step” &gt; choose “Execute Shell” &gt; give any basic shell command. (Eg: free -m)<br> <br> 3. Configure Job to send logs to Elasticsearch,<br> <br>4. In the same job, go to Post-build Actions &gt; Choose “Send Console log to Logstash” Save it.<br> <br> Now click “Build now” and check the logs in Elasticsearch by Chrome browser plugin “Elasticsearch Head”<br> <br> example data,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*FRBDXB45t_SvhYgg.PNG" /><figcaption>Elasticsearch Index Data</figcaption></figure><p>STEP 6 :-<br> <br> Add Index Pattern in Kibana to Analyse the logs,<br> <br> Login to Kibana console &gt; Management &gt; Index Patterns &gt; Create Index Patterns, here we can see the Elasticsearch Index then create it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*SIj7gzUnkSOYcX-D.PNG" /><figcaption>Kibana Dashboard — Create index Pattern</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*EW2HjpdunWUzoymJ.PNG" /><figcaption>Real Time Jenkins Build Logs on to Kibana Dashboard</figcaption></figure><p>That’s all, Successfully exported Jenkins build jobs to Elasticsearch and visualized by Kibana to analyse the Build logs.</p><p>Thank you for being an awesome reader <strong>😃</strong></p><p>Please 👏 👏 the article for giving Positive Vibes to me <strong>😃.</strong></p><p>Check out my other blogs also.</p><ul><li><a href="https://medium.com/@lalit.soni/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c">What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?</a></li><li><a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">What is ELK Stack? Elasticsearch, Logstash and Kibana</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad372429858d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?]]></title>
            <link>https://faun.pub/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/26516b2d186c</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 00:55:46 GMT</pubDate>
            <atom:updated>2022-03-01T11:15:59.256Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/598/0*oURTEqdiBTBKygNf.png" /><figcaption>jenkins — automation tool</figcaption></figure><p>Whenever we talk about DevOps, the first thing came into our mind is Speed. Speed to deploy the application into Pre-prod or Production is important to implement DevOps into any Project.</p><p>If you have read about DevOps then I am very sure you know the term CI/CD Pipeline i.e Continuous Integration and Continuous Delivery/Deployment.</p><p><em>To achieve the speed to deploy the code from Developer to Production we need to do some Automation. For Automation, we need a tool i.e Jenkins.</em></p><p>Jenkins is an Automation tool that is the heart of the CI/CD Pipeline. Every phase of the CI/CD pipeline we can automate using this open-source Automation tool. Some engineers called it Automation Server.</p><p>Jenkins helps automate the parts of <a href="https://en.wikipedia.org/wiki/Software_development">software development</a> related to <a href="https://en.wikipedia.org/wiki/Software_build">building</a>, <a href="https://en.wikipedia.org/wiki/Test_automation">testing</a>, and <a href="https://en.wikipedia.org/wiki/Software_deployment">deploying</a>, facilitating <a href="https://en.wikipedia.org/wiki/Continuous_integration">continuous integration</a> and <a href="https://en.wikipedia.org/wiki/Continuous_delivery">continuous delivery</a></p><h3>Before Jenkins</h3><p>Before DevOps culture, the development team, must Integrate and Test their code manually. Locating and fixing bugs after a test was difficult and time-consuming, which delayed the entire delivery process. The quality of the software has been compensated. Also, time taken for deployment was huge.</p><h3>After Jenkins</h3><p>Jenkin achieves Continuous Integration with the help of well over 1000 plugins, that allows you to build, test and deploy on a continuous basis. Jenkins’s organization can aid firms with fast-track development and life-cycle process. The automation includes a static analysis and requires little to no maintenance once automated and has a built-in GUI for easy updates. Quality has become uncompromised.</p><p>To achieve faster delivery/deployment time we need to use an Automation tool like Jenkins. We can automate any phase of the CI/CD pipeline because every phase i.e builds, test, deployment is connected with the Jenkins. That’s why I called it the heart of the CI/CD pipeline or important to implement DevOps.</p><h3>Why Jenkins is Popular?</h3><ol><li>It is free and Open Source</li><li>Great Community Support available</li></ol><h3>How to Install Jenkins on AWS EC2 Instance or any VM (Virtual Machine) ?</h3><blockquote><em>If you are video person then i have already attached youtube video for Jenkins Installation. Go and Check out.</em></blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fm2iFmo4Os3Y%3Fstart%3D9%26feature%3Doembed%26start%3D9&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dm2iFmo4Os3Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fm2iFmo4Os3Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcad35dd80a653f57c8c027308ec0da2/href">https://medium.com/media/fcad35dd80a653f57c8c027308ec0da2/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*ecm3Q40a9to920Sy" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/366/0*l-JG95spW8aAqWft.png" /><figcaption>Jenkins Installation</figcaption></figure><p>To install Jenkins, we need a VM on any cloud service provider (AWS, GCP, Azure, etc). Also, We can install Jenkins on Local machines but it is advisable to use VM in Cloud.</p><p>Jenkins is developed in JAVA so JAVA is a prerequisite to install and set up Jenkins.</p><ol><li><strong>Install java using the following command</strong></li></ol><blockquote>yum install java* -y</blockquote><p><strong>2. To ensure that your software packages are up to date on your instance, use the following command to perform a quick software update:</strong></p><blockquote><em>sudo yum update –y</em></blockquote><p><strong>3. Add the Jenkins repo using the following command:</strong></p><blockquote><em>sudo wget -O /etc/yum.repos.d/jenkins.repo </em><a href="http://pkg.jenkins-ci.org/redhat/jenkins.repo"><em>http://pkg.jenkins-ci.org/redhat/jenkins.repo</em></a></blockquote><p><strong>4. Import a key file from Jenkins to enable installation from the package:</strong></p><blockquote><em>sudo rpm — import </em><a href="http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key"><em>http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key</em></a></blockquote><p><strong>5. Now Install Jenkins :</strong></p><blockquote><em>sudo yum install jenkins -y</em></blockquote><p><strong>6. Start Jenkins as a service</strong></p><blockquote><em>sudo service jenkins start</em></blockquote><p>Now, you can go on your 8080 port of public ip using a web browser i.e public_ip:8080/. After visiting url, you can see the below screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*1D7pYmQKUgPUNv3v" /><figcaption><a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">ELK Stack</a></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/0*fe9WE2W0mghWWSU6.png" /><figcaption>jenkins — Getting started Page</figcaption></figure><p>Now to unlock Jenkins for the first time you need an initial Password which is stored in this particular directory, Use this command :</p><blockquote><em>sudo cat /var/lib/jenkins/secrets/initialAdminPassword</em></blockquote><p>Paste the password into the Getting started Page of Jenkins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*q7ze2QzAzZ4mSGqf" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/0*gAj8VD4YfzgNbonQ.png" /><figcaption>Initial plugin installation</figcaption></figure><p>Here your installation steps completed and you can start configuring and managing the Jenkins Automation server as per your requirement. Automate the task for your workflow.</p><h4>Jenkins welcome page will be like</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*V3NGIOY6f_tFQjNs" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/0*_KiOBWlED6fj6BSj.png" /><figcaption>jenkins welcome dashboard</figcaption></figure><h3>Thanks for reading the Post!!</h3><p>Check out my other posts related to Latest Tech and DevOps.</p><h3>ELK Stack — Best Opensource Monitoring Tool</h3><h4><a href="https://medium.com/@lalit.soni/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c">ELK Stack</a></h4><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26516b2d186c" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/what-is-jenkins-why-it-is-important-in-devops-how-to-install-jenkins-on-aws-ec2-instance-26516b2d186c">What is Jenkins? Why it is important in DevOps? How to install Jenkins on AWS EC2 Instance?</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is ELK Stack? Elasticsearch, Logstash and Kibana]]></title>
            <link>https://lalit-soni.medium.com/what-is-elk-stack-elasticsearch-logstash-and-kibana-6b26b5b1a79c?source=rss-50016ee8d45d------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b26b5b1a79c</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Lalit Soni - Cloud Engineering | MLOps | DevOps]]></dc:creator>
            <pubDate>Mon, 18 Oct 2021 00:44:27 GMT</pubDate>
            <atom:updated>2021-10-18T00:46:23.006Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/927/0*JsjQGWzOoen5kpc4.png" /></figure><p>source: <a href="https://elastic-stack.readthedocs.io/en/latest/introduction.html">https://elastic-stack.readthedocs.io/</a></p><p>Many Software Applications are deployed on Cloud (i.e AWS, GCP, Azure) which follow the Distributed Architecture. Means Webservers, Database servers, etc are running somewhere in the Cloud on different VMs(Virtual Machines). To monitor the Application and Infrastructure there is a need for a monitoring solution.</p><p>Let me give you one more simple example, Lets consider your application is running on 40 VMs on Cloud and they are generating server logs simultaneously and if you are monitoring those logs on a central monitoring platform (i.e ELK) then quickly you can identify the issue if there is any and troubleshoot it on that particular VM only. This way overhead of the engineer is reduced and troubleshooting happened quickly. That’s why monitoring of the Application and Infrastructure is necessary.</p><p>To manage and maintain the system in healthier way, we need to analyze full system activities, that’s where monitoring solution like ELK Stack came into the picture.</p><p>In this Blogpost, I am going to give you introduction about ELK Stack and some of its features and alternatives of ELK in the market.</p><p>ELK Stack is an Open Source Distributed monitoring solution and Log management solution.</p><p>ELK is the collection of three open source tools i.e</p><ol><li>Elasticsearch</li></ol><p>2. Logstash</p><p>3. Kibana</p><p>These three tools were developed and maintained by a Company named <strong>ELASTIC</strong>. The official website of ELASTIC is <a href="https://www.elastic.co/">https://www.elastic.co/</a> . ELK Stack gives the ability to aggregate logs from all your system and Applications and also create visualizations for Application and Infrastructure Monitoring.</p><p>E stands for <strong>ELASTICSEARCH</strong> which is a search and analytics engine. Also, it is used to store and index the logs. In fact, it is the main component of ELK Stack.</p><p>L<strong> </strong>stands for <strong>LOGSTASH</strong> which is a data ingestion tool which allow users to easily ingest data from variety of sources, transform it and send it to desired destination for storing.</p><p>K<strong> </strong>stands for <strong>KIBANA</strong> which allow user to visualize Data using different charts and visualization techniques to analyze the Logs and Events( A Web Interface to explore the Data present in Elasticsearch).</p><h3>What is Elasticsearch ? — A Real-time search and Analytics tool</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*wl1rY6xTRTsWpJyj" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*8aQeXsSVRC3rQe8w.png" /></figure><p>Source: Elastic.co</p><p>Elasticsearch is a distributed, open-source search and analytics engine built on the Apache Lucene library and developed in JAVA. It works on Structured, Unstructured, Numerical, and Geospatial Data. The Data is stored in the form of a JSON Document.</p><p>Elasticsearch is a NoSQL database. It is distributed, centrally stores your data, allow RESTful searches and this analytical engine is capable of solving different kinds of growing number of use cases. Because of the power of elasticsearch, It has been adopted in search engine platforms for modern web and mobile applications.</p><h3>Features of Elasticsearch -</h3><ul><li>Lightening fast full-text search.</li><li>Security analytics and infrastructure monitoring.</li><li>Can be scaled to thousands of servers and can handle petabytes of data.</li><li>Can be integrated with Kibana to provide real-time visualization of Elasticsearch data for accessing application performance and for monitoring logs and infrastructure metrics data.</li><li>It uses Machine Learning to automatically model the behaviour of your data in real-time.</li></ul><h3>What is Logstash? — Ingesting and Routing your Data</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*I5YW37Hf_7Cb_1Wv" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*own3RVno7jhc2MQj.png" /></figure><p>source: elastic.co</p><p>Logstash is a Data collection pipeline tool that accepts inputs from various sources, executes different transformations, and exports the data to various targets. It collects Data Inputs and Feeds it into Elasticsearch.</p><p>This virtually any type of log that you manage i.e System logs, Webserver logs, Error logs and App logs.</p><p>With Logstash, it’s super easy to take all logs and store them in a central location. The only prerequisite is a java runtime, and it takes just two commands to get Logstash up and running.</p><p>Think of Logstash as a pipeline for event processing i.e it takes precious little time to choose the inputs, configure the filters, and extract the relevant, high-value data from your logs.</p><h3>What is Kibana ? — Visualization Tool</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/60/0*m7EFRFV7CVfxY3bb" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/0*bZ9g9aWkZNgAsg-2.png" /></figure><p>source: Elastic.co</p><p>Kibana is a Data Visualization Dashboard. This tool is helpful for DevOps Engineers, Developers, System Admins, Business Managers, Project Managers, etc. This dashboard offers various Interactive diagrams, Geospatial data, and graphs to visualize.</p><p>It can be used for search, view, and interact with data stored in Elasticsearch directories.</p><p>You can Visualize trends and patterns for data that would otherwise be extremely tedious to read and interpret. Eventually, each business line can make practical use of your data collection as you help them customize their dashboards.</p><h3>Alternative Distributed Log Management Tools: —</h3><ol><li>Splunk</li></ol><p>2. Nagios</p><p>3. Zabbix</p><p>4. Prometheus</p><p>5. LogDNA</p><p>6. Garylog</p><p>7. Fluentd</p><p>Please 👏👏👏👏 the article if you liked it. It will boost my confidence.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b26b5b1a79c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>