<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mamoona Aslam on Medium]]></title>
        <description><![CDATA[Stories by Mamoona Aslam on Medium]]></description>
        <link>https://medium.com/@mamoonaaslam?source=rss-2d1440743bb0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*VeCGH3hO6ZMJJw3HkHMVTw.png</url>
            <title>Stories by Mamoona Aslam on Medium</title>
            <link>https://medium.com/@mamoonaaslam?source=rss-2d1440743bb0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:09:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mamoonaaslam/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Setting Up Kubernetes Dashboard on k3d Cluster]]></title>
            <link>https://medium.com/@mamoonaaslam/setting-up-kubernetes-dashboard-on-k3d-cluster-7bd2e261e42a?source=rss-2d1440743bb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7bd2e261e42a</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[dashboard]]></category>
            <category><![CDATA[k3d]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Mamoona Aslam]]></dc:creator>
            <pubDate>Sun, 29 Sep 2024 19:52:25 GMT</pubDate>
            <atom:updated>2024-09-29T19:57:17.835Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pp6NNrsgtsbGPLVC" /></figure><p>In today’s fast-paced DevOps world, managing Kubernetes clusters efficiently is a must. Having a visual representation of the cluster’s health and activity can make a big difference. This is where the <strong>Kubernetes Dashboard</strong> comes in. It allows you to view your resources, manage workloads, and troubleshoot issues visually. If you’re running a local development environment with Kubernetes, a tool like <strong>k3d</strong> makes it easy to spin up clusters in Docker containers.</p><p>In this guide, we will walk through the steps to set up a <strong>Kubernetes Dashboard</strong> on a <strong>k3d cluster</strong>. We’ll cover everything, from installing the required tools like Docker, k3d, and kubectl, to creating and accessing the dashboard itself.</p><h3>System Requirements</h3><p>Before jumping into the installation, let’s review the basic system requirements. This ensures everything runs smoothly once we start deploying the Kubernetes cluster and the dashboard.</p><ul><li><strong>Operating System</strong>: We’re assuming a Linux-based system for this guide. Ubuntu 24.04 is used as a reference, but most modern Linux distributions should work.</li><li><strong>Memory</strong>: At least 4 GB of RAM. Kubernetes clusters, even small ones, require memory to run smoothly.</li><li><strong>Disk Space</strong>: Allocate at least 10 GB for Docker images, the cluster, and other related files.</li></ul><p>On top of this, you’ll need three core tools installed on your machine:</p><ol><li><strong>Docker</strong>: Container engine that k3d uses to run the Kubernetes nodes.</li><li><strong>k3d</strong>: A lightweight wrapper to run Kubernetes clusters inside Docker containers.</li><li><strong>kubectl</strong>: The CLI for interacting with Kubernetes clusters.</li></ol><h3>Step 1: Installing Docker</h3><p>Docker is the foundation of our setup, as it allows us to run containers on our system. k3d uses Docker containers to spin up Kubernetes clusters quickly, making it essential for our environment.</p><h3>Installing Docker on Ubuntu</h3><ol><li><strong>Uninstall Old Versions</strong> (if any): If you have older versions of Docker installed, remove them to avoid conflicts:</li></ol><pre>sudo apt-get remove docker docker-engine docker.io containerd runc</pre><p>2. <strong>Install Docker’s Dependencies</strong>: Docker needs a few dependencies to run. Install them using:</p><pre>sudo apt-get update<br>sudo apt-get install ca-certificates curl gnupg lsb-release</pre><p>3. <strong>Add Docker’s GPG Key</strong>: This ensures the integrity of Docker’s packages when they are downloaded:</p><pre>sudo mkdir -m 0755 -p /etc/apt/keyrings<br>curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo tee /etc/apt/keyrings/docker.gpg &gt; /dev/null</pre><p>4. <strong>Set Up the Docker Repository</strong>: We now need to configure the Docker repository so we can install it:</p><pre>echo \<br>&quot;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \<br>$(lsb_release -cs) stable&quot; | sudo tee /etc/apt/sources.list.d/docker.list &gt; /dev/null</pre><p>5. <strong>Install Docker Engine</strong>: With the repository set up, install Docker using:</p><pre>sudo apt-get update<br>sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin</pre><p>6. <strong>Start Docker</strong>: Once installed, start the Docker service:</p><pre>sudo systemctl start docker<br>sudo systemctl enable docker</pre><p>7. <strong>Verify Installation</strong>: To confirm Docker is installed correctly, run:</p><pre>docker --version</pre><p>If all went well, Docker should now be up and running on your system.</p><h3>Step 2: Installing k3d</h3><p>k3d is a fantastic tool that wraps <strong>k3s</strong> (a lightweight Kubernetes distribution) inside Docker containers, allowing you to run Kubernetes clusters locally without any hassle. Let’s install it.</p><h3>Installing k3d</h3><ol><li><strong>Install k3d</strong> using the official script</li></ol><pre>curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash</pre><p>2. <strong>Verify Installation</strong>: Check if k3d was installed correctly by running:</p><pre>k3d --version</pre><p>And with that, you’re ready to create and manage Kubernetes clusters locally with k3d.</p><p>And with that, you’re ready to create and manage Kubernetes clusters locally with k3d.</p><h3>Step 3: Installing kubectl</h3><p>kubectl is the command-line tool that allows you to interact with your Kubernetes clusters. It’s a must-have if you’re working with Kubernetes.</p><h3>Installing kubectl on Ubuntu</h3><ol><li><strong>Install kubectl</strong> using the package manager:</li></ol><pre>sudo apt-get update<br>sudo apt-get install -y kubectl</pre><p>2. <strong>Verify Installation</strong>: Make sure kubectl is working by running:</p><pre>kubectl version --client</pre><p>Now that we’ve installed Docker, k3d, and kubectl, we can proceed to set up the Kubernetes Dashboard.</p><h3>Step 4: Setting Up Kubernetes Dashboard</h3><h3>Enable IP Forwarding</h3><p>Before we start creating the cluster, it’s important to enable <strong>IP forwarding</strong>. Kubernetes requires this for proper network routing within the cluster. Enable it with the following command:</p><pre>echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward</pre><p>This command ensures that your machine can forward network traffic, which is necessary for the Kubernetes networking to function properly.</p><h3>Step 5: Create a k3d Cluster</h3><p>We will now create a Kubernetes cluster using k3d. In this example, we’re going to create a cluster named dashboard-cluster with two agent nodes and a load balancer.</p><pre>k3d cluster create dashboard-cluster --api-port 6550 -p &quot;8080:80@loadbalancer&quot; --agents 2</pre><ul><li>--api-port 6550: Exposes the Kubernetes API server on port 6550.</li><li>-p &quot;8080:80@loadbalancer&quot;: Maps your local machine&#39;s port 8080 to port 80 of the cluster&#39;s load balancer.</li><li>--agents 2: Creates two worker nodes (agents) in the cluster.</li></ul><p>This command spins up a lightweight Kubernetes cluster inside Docker containers, making it perfect for local development.</p><h3>Step 6: Deploy Kubernetes Dashboard</h3><p>With the cluster up and running, the next step is to deploy the <strong>Kubernetes Dashboard</strong>. The dashboard allows you to visualize and manage the Kubernetes cluster from a web interface.</p><p>You can deploy it by applying the official YAML file:</p><pre>kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml</pre><p>This command will create all the necessary Kubernetes objects, including the dashboard service, pods, and deployments. The dashboard will allow you to view the cluster’s health, manage resources, and troubleshoot issues visually.</p><h3>Step 7: Create an Admin Service Account</h3><p>To log into the dashboard, you’ll need an admin-level service account. Let’s create one:</p><ol><li><strong>Create the service account</strong>:</li></ol><pre>kubectl create serviceaccount dashboard-admin</pre><p>2. <strong>Bind the service account to the cluster-admin role</strong>:</p><pre>kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=default:dashboard-admin</pre><p>These commands create a service account with administrative privileges, which we’ll use to log in to the dashboard later.</p><h3>Step 8: Retrieve Access Token</h3><p>To authenticate yourself when logging into the dashboard, you’ll need a token associated with the service account we just created. Generate it with the following command:</p><pre>kubectl create token dashboard-admin</pre><p>Copy this token and keep it safe — you’ll need it in the next step to access the dashboard.</p><h3>Step 9: Access the Kubernetes Dashboard</h3><p>Finally, we need to expose the dashboard so that it’s accessible via our local machine. You can do this using port forwarding:</p><pre>kubectl port-forward -n kubernetes-dashboard service/kubernetes-dashboard 8443:443 --address 0.0.0.0</pre><p>This command exposes the dashboard on port 8443. You can now access it by navigating to https://localhost:8443 in your browser. When prompted, enter the token you generated earlier.</p><p>Congratulations! You now have a fully functional Kubernetes dashboard running on your k3d cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7aURYyHYwSHbbt2YC-Is6g.png" /></figure><h3>Troubleshooting</h3><p><strong>Common issues and solutions</strong>:</p><ul><li><strong>IP Forwarding Issues</strong>: If the cluster network doesn’t function properly, ensure that IP forwarding is enabled by running cat /proc/sys/net/ipv4/ip_forward. It should return 1. If not, re-enable it using the command from <a href="#step-4-setting-up-kubernetes-dashboard">Step 4</a>.</li><li><strong>Port Conflicts</strong>: If port forwarding fails, ensure that no other service is using ports 8443 or 8080. Use lsof -i :PORT_NUMBER to check for conflicts.</li><li><strong>Service Not Starting</strong>: If the dashboard doesn’t load, check the pod logs for errors using:</li></ul><pre>kubectl logs -n kubernetes-dashboard POD_NAME</pre><h3>Conclusion</h3><p>Setting up a <strong>Kubernetes Dashboard</strong> using <strong>k3d</strong> offers a simple yet powerful way to visually manage your Kubernetes clusters. In this guide, we’ve walked through installing Docker, k3d, and kubectl, as well as creating a local cluster and deploying the dashboard. By following these steps, you now have a streamlined local environment that’s ideal for both development and testing. The Kubernetes Dashboard provides a rich interface for monitoring resources, managing workloads, and troubleshooting issues, making it an essential tool for anyone working with Kubernetes. Whether you’re a beginner looking to get started with Kubernetes or an experienced professional needing a lightweight local cluster, this setup has you covered.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7bd2e261e42a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Monitor Microservices Effectively with Grafana and Loki]]></title>
            <link>https://medium.com/@mamoonaaslam/how-to-monitor-microservices-effectively-with-grafana-and-loki-c2c73614258d?source=rss-2d1440743bb0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2c73614258d</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[log-aggregation]]></category>
            <category><![CDATA[grafana-loki]]></category>
            <category><![CDATA[microservice-architecture]]></category>
            <dc:creator><![CDATA[Mamoona Aslam]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 00:46:13 GMT</pubDate>
            <atom:updated>2024-09-19T00:46:13.727Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ww6z3iVWkvdcq3SX" /></figure><h3>Introduction</h3><p>Microservices offer a flexible approach to software architecture, allowing independent deployment, scalability, and maintenance of individual components. However, managing and monitoring these services presents unique challenges. Centralized logging is a vital strategy to aggregate logs from various microservices into a single, searchable location. Grafana Loki, a log aggregation system, is an excellent tool for this purpose. It integrates seamlessly with Grafana, allowing visualization of logs alongside metrics and traces.</p><p>This tutorial will provide a walk through setting up a centralized logging solution using Grafana Loki, Promtail, and Grafana for an e-commerce application composed of three microservices: Product Service, Order Service, and User Service running on Docker containers.</p><h3>System Requirements</h3><p>Before diving into the setup, let’s ensure the environment meets the following system requirements:</p><ul><li><strong>Processor</strong>: Dual-core processor (2 GHz or higher)</li><li><strong>RAM</strong>: Minimum 4 GB (8 GB recommended)</li><li><strong>Operating System</strong>: Debian 12 / Ubuntu 20.04 or later</li><li><strong>Virtualization</strong>: VirtualBox 7.0 or later</li><li><strong>Vagrant</strong>: Version 2.4 or later</li><li><strong>Docker</strong>: Version 27.2 or later</li><li><strong>Docker Compose</strong>: Version 2.20 or later</li><li><strong>Ansible</strong>: Version 2.16 or later</li></ul><h3>Understanding the Architecture</h3><p>In this setup, there is an e-commerce platform with three microservices:</p><ol><li><strong>Product Service</strong>: Manages product information.</li><li><strong>Order Service</strong>: Handles order processing.</li><li><strong>User Service</strong>: Manages user accounts and authentication.</li></ol><p>Each service is containerized using Docker, and all the containers are orchestrated with Docker Compose. Promtail is configured to collect logs from the microservices and send them to Loki. Grafana is used to visualize these logs, providing a centralized view of the application’s health and performance.</p><p>The architecture looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O0iIedjfiXjK0OdNYUyK_g.png" /><figcaption>Microservices Monitoring and Logging using Loki and Grafana</figcaption></figure><h3>Setting Up the Environment</h3><p>We will create a <strong>Vagrant VM</strong> with <strong>Ubuntu</strong> and use <strong>Ansible</strong> for provisioning. The microservices will be containerized using <strong>Docker</strong>, and we’ll use <strong>Docker Compose</strong> to orchestrate them. <strong>Grafana </strong>will be set up to visualize metrics and logs, while <strong>Loki</strong> will aggregate the logs from the microservices.</p><h3><strong>Vagrant Configuration</strong></h3><p>Vagrant is used to set up a virtual machine (VM) that acts as the host for the microservices. Vagrant allows to create a consistent environment for development and testing.</p><p>Create a Vagrantfile with the following configuration:</p><pre>Vagrant.configure(&quot;2&quot;) do |config|<br>  config.vm.box = &quot;ubuntu/focal64&quot;<br><br>  # Define a hostname for your VM<br>  config.vm.hostname = &quot;microservices-vm&quot;<br><br>  # Port forwarding<br>  config.vm.network &quot;forwarded_port&quot;, guest: 3000, host: 3000  # Grafana<br>  config.vm.network &quot;forwarded_port&quot;, guest: 3100, host: 3100  # Loki<br>  config.vm.network &quot;forwarded_port&quot;, guest: 5001, host: 5001  # Product Service<br>  config.vm.network &quot;forwarded_port&quot;, guest: 5002, host: 5002  # Order Service<br>  config.vm.network &quot;forwarded_port&quot;, guest: 5003, host: 5003  # User Service<br><br>  config.vm.provider &quot;virtualbox&quot; do |vb|<br>    vb.memory = &quot;4096&quot;<br>    vb.cpus = 2<br>  end<br><br>  # Sync a folder from your host machine to the guest VM<br>  config.vm.synced_folder &quot;./microservices&quot;, &quot;/home/vagrant/microservices&quot;, owner: &quot;vagrant&quot;, group: &quot;vagrant&quot;, mount_options: [&quot;dmode=775,fmode=664&quot;]<br><br>    # Ansible provisioning<br>  config.vm.provision &quot;ansible_local&quot; do |ansible|<br>    ansible.playbook = &quot;playbook.yml&quot;<br>    #ansible.provisioning_path = &quot;/vagrant&quot;<br>    ansible.inventory_path = &quot;inventory&quot;<br>    # Run as root<br>    ansible.become = true<br>    # Limit the Ansible run to this single VM<br>    ansible.limit = &quot;all&quot;<br>  end<br><br>  # Shell provisioning for Docker and Docker Compose<br>  config.vm.provision &quot;shell&quot;, inline: &lt;&lt;-SHELL<br>    # Update package list and install Docker<br>    sudo apt-get update<br>    sudo apt-get install -y docker.io docker-compose<br><br>    # Start Docker service<br>    sudo systemctl start docker<br>    sudo systemctl enable docker<br><br>    # Change to the directory with Docker Compose file<br>    cd /home/vagrant/microservices/dockerfiles<br>    # Run Docker Compose<br>    sudo docker-compose up --build -d<br>  SHELL<br>end</pre><p>This configuration:</p><ul><li>Uses the Ubuntu 20.04 image for the VM.</li><li>Allocates 4 GB of RAM and 2 CPU cores.</li><li>Maps the host folder containing microservices to the guest VM to ensure code synchronization.</li><li>Forwards necessary ports for Grafana, Loki, and the microservices.</li><li>Installs Docker and Docker Compose.</li><li>Runs an Ansible playbook to provision and configure the necessary services i.e Promtail, Loki and Grafana.</li></ul><h3>Ansible Playbook</h3><p>Ansible playbook automates the installation and configuration of Grafana, Loki, and Promtail on the Virtual Machine.</p><p>Create a playbook.yml with the following content:</p><pre>---<br>- hosts: local<br>  become: yes<br><br>  roles:<br>    - common<br>    - loki<br>    - promtail<br>    - grafana<br><br>  tasks:<br>    - name: Reload systemd and start services<br>      systemd:<br>        daemon_reload: yes<br><br>    - name: Start and enable Loki and Promtail services<br>      systemd:<br>        name: &quot;{{ item }}&quot;<br>        state: started<br>        enabled: yes<br>      loop:<br>        - loki<br>        - promtail<br><br>    - name: Wait for Grafana to be ready<br>      wait_for:<br>        port: 3000<br>        delay: 10<br>        timeout: 300<br><br>    - name: Configure Grafana datasource<br>      uri:<br>        url: http://localhost:3000/api/datasources<br>        method: POST<br>        body_format: json<br>        body:<br>          name: Loki<br>          type: loki<br>          url: http://localhost:3100<br>          access: proxy<br>        user: admin<br>        password: admin<br>        force_basic_auth: yes<br>        status_code: 200<br>      ignore_errors: yes<br><br>    - name: Get Loki data source UUID<br>      uri:<br>        url: &quot;http://localhost:3000/api/datasources&quot;<br>        user: admin<br>        password: admin<br>        force_basic_auth: yes<br>      register: datasources<br><br>    - name: Set Loki UUID fact<br>      set_fact:<br>        loki_uid: &quot;{{ datasources.json | json_query(&#39;[?type==`loki`].uid&#39;) | first }}&quot;<br>    <br>    - name: Create Grafana dashboard JSON with Loki UID<br>      template:<br>        src: dashboard.json.j2<br>        dest: microservice.json  <br><br>    - name: Import Grafana dashboard Microservices<br>      community.grafana.grafana_dashboard:<br>        grafana_url: http://localhost:3000<br>        grafana_user: &quot;admin&quot;<br>        grafana_password: &quot;admin&quot;<br>        state: present<br>        commit_message: Updated by ansible<br>        overwrite: true<br>        path: microservice.json</pre><p>This playbook:</p><ul><li>Installs and configures Loki and Promtail as services.</li><li>Configures Grafana with Loki as a data source.</li><li>Imports a Grafana dashboard for visualizing the microservices’ logs.</li></ul><h3>Implementing the Microservices</h3><p>Each microservice in our application is a simple Flask application that logs structured data using structlog. Here’s an example of the Order Service:</p><pre>import logging, sys<br>from flask import Flask, jsonify, request<br>import structlog<br><br># Set up  logging<br>logging.basicConfig(level=logging.INFO,<br>                    stream=sys.stdout,<br>                    format=&#39;%(asctime)s - %(name)s - %(levelname)s - %(message)s&#39;)<br><br># Set up structlog to use both console and file logging<br>structlog.configure(<br>    processors=[<br>        structlog.stdlib.add_log_level,<br>        structlog.stdlib.add_logger_name,<br>        structlog.processors.TimeStamper(fmt=&quot;iso&quot;),<br>        structlog.stdlib.ProcessorFormatter.wrap_for_formatter,<br>    ],<br>    logger_factory=structlog.stdlib.LoggerFactory(),<br>    wrapper_class=structlog.stdlib.BoundLogger,<br>    cache_logger_on_first_use=True,<br>)<br><br>logger = structlog.get_logger()<br><br><br>app = Flask(__name__)<br><br>@app.route(&#39;/order&#39;, methods=[&#39;POST&#39;])<br>def create_order():<br>    order_data = request.json<br>    logger.info(&quot;Processing order&quot;, service=&quot;order_service&quot;, order_id=order_data.get(&#39;id&#39;))<br>    logging.info(f&quot;Processing order with ID: {order_data.get(&#39;id&#39;)}&quot;)<br>    # Simulate order processing<br>    return jsonify({&quot;status&quot;: &quot;Order processed successfully&quot;})<br><br>if __name__ == &#39;__main__&#39;:<br>    app.run(host=&#39;0.0.0.0&#39;, port=5002)</pre><p>Similar applications are created for the Product Service and User Service.</p><h3>Dockerfile for Microservices</h3><p>We use Docker to containerize our services. Each microservice has a corresponding Dockerfile.Here’s an example Dockerfile for the Order Service:</p><pre># Use an official Python runtime as a parent image<br>FROM python:3.9-slim<br><br># Set the working directory<br>WORKDIR /app<br><br># Create the logs directory<br>RUN mkdir -p /app/logs<br><br># Copy the requirements file into the container<br>COPY  services-py/requirements.txt .<br><br># Install any dependencies<br>RUN pip3 install --no-cache-dir -r requirements.txt<br><br># Copy the Python script into the container<br>COPY  services-py/order_service.py .<br><br># Expose the port the app runs on<br>EXPOSE 5002<br><br># Define the command to run the application<br>CMD [&quot;python&quot;, &quot;-u&quot;, &quot;order_service.py&quot;]</pre><p>The service runs a Flask app. The application is containerized, with the necessary dependencies installed via pip</p><h3>Microservices Setup with Docker Compose</h3><p>The docker-compose.yml file orchestrates all the services:</p><pre>version: &#39;3.3&#39;<br><br>services:<br>  product-service:<br>    container_name: product-service<br>    build:<br>      context: .<br>      dockerfile: Dockerfile.product<br>    ports:<br>      - &quot;5001:5001&quot;<br>    labels:<br>      logging: &quot;promtail&quot;<br>      logging_jobname: &quot;containerlogs&quot;<br><br>  order-service:<br>    container_name: order-service<br>    build:<br>      context: .<br>      dockerfile: Dockerfile.order<br>    ports:<br>      - &quot;5002:5002&quot;<br>    labels:<br>      logging: &quot;promtail&quot;<br>      logging_jobname: &quot;containerlogs&quot;<br><br>  user-service:<br>    container_name: user-service<br>    build:<br>      context: .<br>      dockerfile: Dockerfile.user<br>    ports:<br>      - &quot;5003:5003&quot;<br>    labels:<br>      logging: &quot;promtail&quot;<br>      logging_jobname: &quot;containerlogs&quot;<br></pre><p>Each service (Product, Order, and User) is defined in Docker Compose. They are exposed on different ports and labeled for Promtail to collect logs.</p><h3>Running the Setup</h3><p>Once everything is set up, simply running vagrant up will automate the following steps:</p><p><strong>Virtual Machine Creation</strong>:</p><ul><li>A VM is provisioned using Vagrant, with Ubuntu as the base OS.</li><li>Ports are forwarded for accessing services like Grafana, Loki, and the microservices from your host machine.</li></ul><p><strong>Ansible Provisioning</strong>:</p><ul><li>Ansible is automatically triggered to install <strong>Docker</strong> and <strong>Docker Compose</strong>.</li><li>Ansible will also set up the <strong>Grafana</strong>, <strong>Loki</strong>, and <strong>Promtail</strong> services.</li></ul><p><strong>Microservices Setup</strong>:</p><ul><li><strong>Docker Compose</strong> is executed to spin up the three microservices: <strong>Product Service</strong>, <strong>Order Service</strong>, and <strong>User Service</strong>.</li><li>These services run as Python Flask applications inside Docker containers.</li><li>Each microservice is exposed on its respective port (5001, 5002, 5003) for external access.</li></ul><p><strong>Loki &amp; Promtail Setup</strong>:</p><ul><li><strong>Loki</strong> is installed and configured as a central log aggregator.</li><li><strong>Promtail</strong> is set up to collect logs from the Docker containers. It identifies logs based on container labels (e.g., product-service, order-service) and ships them to Loki.</li></ul><p><strong>Log Collection with Promtail</strong>:</p><ul><li>Promtail collects logs from each of the running microservice containers.</li><li>These logs are forwarded to Loki, where they are indexed and made available for querying and visualization.</li></ul><p><strong>Grafana Configuration</strong>:</p><ul><li>Grafana is configured to use <strong>Loki</strong> as a data source for logs.</li><li>A predefined dashboard is imported into Grafana to visualize the logs and metrics from your microservices.</li></ul><p><strong>Access Grafana</strong>:</p><ul><li>After everything is set up, access Grafana at http://localhost:3000 (default login: admin / admin).</li><li>You can visualize the logs from each microservice and filter logs based on service, timestamp, or error level.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yhkd9VTX8uVTIzT_JKCyBA.png" /></figure><h3>Conclusion</h3><p>This setup allows you to easily monitor your microservices using <strong>Grafana</strong> and <strong>Loki</strong>, with <strong>Promtail</strong> handling real-time log collection. It automates the entire process of VM creation, microservice deployment, and log aggregation, providing you with a complete, centralized logging solution and full visibility into your microservices’ health and performance.</p><h3>Source Code</h3><p>If you’d like to refer to the code used in this setup, you can find the complete project on GitHub at:</p><p><a href="https://github.com/mamoona-aslam/microservices-monitoring-grafana-loki">GitHub - mamoona-aslam/microservices-monitoring-grafana-loki: A practical setup for monitoring microservices with Grafana and Loki using Vagrant, Ansible, Docker, and Promtail for centralized log aggregation and visualization.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2c73614258d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>