<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anjul Sahu on Medium]]></title>
        <description><![CDATA[Stories by Anjul Sahu on Medium]]></description>
        <link>https://medium.com/@anjuls?source=rss-808537596ad5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OCcqFj-W7HRcdqE4hhlQ_Q.jpeg</url>
            <title>Stories by Anjul Sahu on Medium</title>
            <link>https://medium.com/@anjuls?source=rss-808537596ad5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:35:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anjuls/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Running PostgreSQL on Kubernetes: CloudNativePG and Commercial Support Options]]></title>
            <link>https://anjuls.medium.com/running-postgresql-on-kubernetes-cloudnativepg-and-commercial-support-options-885ea7cbafc7?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/885ea7cbafc7</guid>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[cncf]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Mon, 13 Apr 2026 04:12:25 GMT</pubDate>
            <atom:updated>2026-04-13T04:12:25.221Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is CloudNativePG?</h3><p>CloudNativePG (CNPG) is an open-source Kubernetes operator built specifically for managing the full lifecycle of PostgreSQL database clusters. It was originally developed by EDB (EnterpriseDB) and donated to the Cloud Native Computing Foundation (CNCF), where it became a CNCF sandbox project in 2022 and later a CNCF incubating project — a sign of growing adoption and community trust.</p><p>The project exists because running stateful workloads like databases on Kubernetes has historically been painful. Kubernetes was designed with stateless applications in mind, and databases bring additional complexity: persistent storage, replication, failover, backups, and connection management all need careful handling. CNPG addresses this by encoding PostgreSQL operational knowledge directly into the operator, so teams don’t have to script their way through every edge case manually.</p><h3>Why CloudNativePG Matters</h3><p>Before CNPG, teams had a few options for running PostgreSQL in production: managed cloud services like AWS RDS or Google Cloud SQL, self-managed instances on VMs, or a patchwork of community Kubernetes operators with varying levels of maturity. None of these options were ideal for teams that wanted both operational control and cloud-native infrastructure.</p><p>CNPG changes that equation. It brings PostgreSQL into the Kubernetes-native world properly — not by wrapping an existing deployment in containers, but by treating the database cluster as a first-class Kubernetes object.</p><h3>Key Features of CloudNativePG</h3><p><strong>Primary/Replica Management:</strong> CNPG handles streaming replication between primary and replica instances automatically. It uses PostgreSQL’s native replication rather than any middleware layer, which keeps things predictable and debuggable.</p><p><strong>Automated Failover:</strong> When a primary node fails, CNPG promotes the most up-to-date replica automatically and updates the service endpoints — no manual intervention needed. The operator continuously monitors replication lag to determine the best candidate for promotion.</p><p><strong>Declarative Configuration:</strong> You define your PostgreSQL cluster as a Kubernetes custom resource (a Cluster object). Want three replicas, specific resource limits, a particular PostgreSQL version, and custom postgresql.conf parameters? That all goes in one YAML file, versioned in Git like everything else.</p><p><strong>Backup and Recovery:</strong> CNPG supports continuous WAL archiving and scheduled base backups to object storage (S3, GCS, Azure Blob). Point-in-time recovery is built in, so you can restore a database cluster to any specific moment — not just the last backup.</p><p><strong>Connection Pooling:</strong> The operator integrates with PgBouncer for connection pooling, which is important for applications that open many short-lived connections. This is managed through a Pooler custom resource alongside your cluster.</p><p><strong>Rolling Updates:</strong> PostgreSQL version upgrades and configuration changes are applied using a rolling strategy — replicas are updated first, then the primary fails over to an updated replica, minimizing downtime.</p><p><strong>Pod Disruption Budgets and Anti-Affinity:</strong> CNPG automatically configures pod disruption budgets and anti-affinity rules to ensure replicas are spread across nodes and that planned maintenance doesn’t take down your entire database cluster.</p><p><strong>Monitoring:</strong> The operator exposes Prometheus-compatible metrics out of the box, covering replication lag, transaction throughput, connection counts, and more.</p><h3>The Reality of Running CNPG in Production</h3><p>Getting CNPG installed and running a test cluster is reasonably straightforward. Running it in production at scale is a different matter. The operator handles a lot, but you still need solid judgment around storage class selection, network policies, backup verification, resource sizing, and disaster recovery testing.</p><p>This is where commercial support comes in — not because the project is immature, but because production databases carry real risk and most engineering teams have other things to focus on.</p><h3>Commercial Support Options for CloudNativePG</h3><h3>1. CloudRaft</h3><p>CloudRaft is a support provider that focuses specifically on databases running on Kubernetes, with CNPG being one of their core offerings. What differentiates them from most vendors is that their team works across both layers: they understand PostgreSQL internals and they understand Kubernetes infrastructure. That combination is less common than you’d expect.</p><p>In practice, this means they can help with things like tuning your storage class for database I/O patterns, configuring anti-affinity rules correctly for your specific cluster topology, or debugging a replication issue that’s partly a Kubernetes networking problem and partly a PostgreSQL configuration issue.</p><p>They also work with adjacent technologies — ClickHouse, TimescaleDB — which matters if your data infrastructure spans more than just standard PostgreSQL. For teams consolidating vendor relationships, <strong>having one support partner across multiple database technologies, Kubernetes and DevOps reduces overhead</strong>.</p><p>CloudRaft has particular experience migrating workloads onto CNPG from managed services like RDS or Cloud SQL, and from traditional VM-based deployments. If reducing cloud provider lock-in is part of your motivation for adopting CNPG, that migration path is where a lot of the risk concentrates, and having experienced help is worthwhile.</p><p>Their pricing is positioned to be accessible for teams that don’t have a big enterprise software budget, though you’d want to talk to them directly about specifics.</p><p>→ <a href="https://www.cloudraft.io/cloudnativepg-commercial-support-for-enterprises">CloudRaft CNPG Support</a></p><h3>2. EDB (EnterpriseDB)</h3><p>EDB created CloudNativePG, and their engineers are among the most active contributors to the project. If you want support from people who literally wrote large portions of the operator, EDB is the obvious choice.</p><p>The tradeoff is that EDB operates at enterprise scale and pricing. They’re a good fit for large organizations that are already in the EDB ecosystem or have procurement processes that favor established vendors. Smaller teams may find their engagement model and cost structure harder to justify.</p><h3>3. Other Providers</h3><p>Several other companies are listed on the official CNPG support page and offer solid PostgreSQL expertise:</p><p><strong>CYBERTEC</strong> has a strong reputation for deep PostgreSQL performance tuning and offers global support coverage.</p><p><strong>Dalibo</strong> is a well-regarded European PostgreSQL consultancy with a genuine open-source focus.</p><p><strong>pgEdge</strong> is worth looking at if your use case involves distributed PostgreSQL or edge deployments — a more specialized niche.</p><p><strong>Axians and B1 Systems</strong> are broader infrastructure providers that include CNPG support as part of wider managed services offerings.</p><p>More providers are listed on the <a href="https://cloudnative-pg.io/support/">CloudNativePG Support page</a>.</p><h3>Choosing the Right Support Partner</h3><p>The right choice depends on a few practical factors: how Kubernetes-centric your infrastructure is, whether you need help with migration, what your budget looks like, and whether you have other database technologies that would benefit from shared support coverage.</p><p>For teams that are <strong>Kubernetes-first and want a partner with genuine depth across both the database and orchestration layers</strong>, <a href="https://www.cloudraft.io/cloudnativepg-commercial-support-for-enterprises">CloudRaft</a> is worth evaluating seriously. For large enterprises with existing EDB relationships or a preference for the operator’s original maintainers, EDB makes sense. <strong>For European teams </strong>or those with specialized distributed database needs, Dalibo and CYBERTEC respectively are strong options.</p><p>CloudNativePG is a mature, well-designed project and the community around it is active. Whichever support path you choose, the operator itself gives you a solid foundation to build on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=885ea7cbafc7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build AI Cloud]]></title>
            <link>https://anjuls.medium.com/how-to-build-ai-cloud-3d4ca915df90?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d4ca915df90</guid>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Wed, 27 Mar 2024 18:41:45 GMT</pubDate>
            <atom:updated>2024-03-27T18:41:45.201Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8rlSYJj0AIoXDECJWvKUHA.png" /></figure><h3>Introduction</h3><p>The field of artificial intelligence (AI) has experienced remarkable growth and evolution in recent years, driven by advancements in machine learning techniques, increased computational power, and the availability of vast amounts of data. One of the most significant developments in this domain has been the emergence of large language models (LLMs), which have demonstrated impressive capabilities in natural language processing tasks such as text generation, summarization, and question-answering.</p><p>As LLMs and other AI models continue to grow in size and complexity, the demand for powerful hardware accelerators, such as B200, NVIDIA’s H100, and GH200 GPUs, as well as solutions from companies like Cerebras (WSE-3) and Groq, has skyrocketed.</p><p>This growth in AI capabilities has been accompanied by a surge in investments from governments, venture capitalists, and individual investors, driven by the promise of better returns on investment (ROI) in this burgeoning field and taking humanity to the next level with <a href="https://openai.com/blog/planning-for-agi-and-beyond">AGI</a>. The democratization of AI, which aims to make these advanced technologies accessible to a broader range of users and organizations, has also gained significant traction.</p><p>Recent news of substantial investments in AI and GPU cloud services underscores the growing recognition of the importance of these technologies. For example, the Indian government <a href="https://pib.gov.in/PressReleaseIframePage.aspx?PRID=2012357">announced</a> the investment of 10,000 GPUs to build the sovereign AI cloud supercomputer. Similarly, investors like Nat &amp; Friedman running their own AI clusters such as <a href="https://andromeda.ai/">Andromeda</a> and <a href="https://gpulist.ai/">so on</a>.</p><p>In this article, we will explore more about AI Cloud, why we need one in more detail, and how to build AI Cloud.</p><h3>What is AI Cloud?</h3><p>AI Cloud refers to a cloud computing infrastructure specifically designed and optimized for the deployment, management, and scaling of AI and machine learning workloads. It combines the power of cloud computing with specialized hardware accelerators, such as GPUs and other AI-focused chips, as well as software tools and frameworks tailored for AI applications. It is different from hyper scalers such as AWS, Google Cloud, or Azure in way that it is meant for one purpose which is to best serve AI workload. They are meant to be cost-effective, particularly for large distributed training jobs, and they intend to democratize AI by simplification.</p><p>Following are the functionalities of the AI cloud that one should expect:</p><ul><li>Ability to manage data that will be used for training purposes. It includes data collection, ingestion, analysis, and curation.</li><li>Model training and evaluation</li><li>Deployment of model (inferencing)</li><li>Hypertuning</li><li>Model monitoring</li><li>Security</li></ul><h3>What is the need for a specialized AI Cloud?</h3><p>The benefits and use cases of AI Cloud highlight the increasing demand for such specialized cloud services. Here are some key advantages and applications:</p><h3>Benefits</h3><ol><li><strong>Sovereign AI Cloud</strong>: AI Cloud can offer enhanced data sovereignty and control, allowing organizations to keep their data and AI operations within their secure cloud environment, adhering to local regulations and privacy requirements.</li><li><strong>Access to cutting-edge models</strong>: While open-source AI models are valuable, the most advanced and powerful models are often closed-source and proprietary. AI Cloud providers can offer access to these state-of-the-art models, enabling organizations to leverage their capabilities without the need for extensive in-house development.</li><li><strong>Lack of operational skills</strong>: Managing AI infrastructure can be complex and resource-intensive, requiring specialized skills in areas such as GPU optimization, model deployment, and scalability. AI Cloud services can abstract away much of this complexity, enabling organizations to focus on their core AI initiatives without the need for extensive in-house operational expertise.</li><li><strong>Reducing costs and improving security</strong>: By leveraging the economies of scale and shared resources of cloud providers, organizations can significantly reduce their AI infrastructure costs while benefiting from enhanced security measures and best practices implemented by cloud providers.</li><li><strong>AI + Cloud Native</strong>: AI Cloud services are often designed with cloud-native principles in mind, enabling seamless integration with other cloud services and facilitating the development of scalable, resilient, and distributed AI applications. You can read more about how cloud native technologies accelerate AI <a href="https://www.cloudraft.io/blog/intersection-of-cloud-native-and-ai">here</a>.</li><li><strong>Democratization of AI</strong>: By lowering the barriers to entry and providing accessible AI resources, AI Cloud services contribute to the democratization of AI, enabling smaller organizations and individuals to leverage these powerful technologies.</li><li><strong>Supply chain issues and GPU availability</strong>: With the increasing demand for GPUs and other specialized hardware, AI Cloud services can help organizations overcome supply chain challenges by innovative GPU scheduling and resource-sharing techniques and ensure access to the necessary compute resources.</li></ol><h3>Use Cases</h3><ol><li>GPU Cloud for Datacenters: AI Cloud services can help datacenters build and provide innovative solutions for their customers so that can maximize their hardware investment and also provide value to the end customers. Various traditional data centers have a plethora of computing but the software part has been lagging. Bringing cloud native technologies to build a stack that can compete with larger cloud players is possible by providing similar or certain categories, a much better user experience, and pricing.</li><li>Large enterprises building internal AI Cloud Platforms: Many large enterprises are developing their own internal AI Cloud Platforms to address specific use cases and requirements such as security, compliances, and IP protection. Another factor to consider here is the cost, with AI, the cloud cost can skyrocket. By utilizing cloud native and open source stack, combined with platform engineering, enterprises can give the best experience to developers and internal stakeholders.</li><li>Air-gapped environments: Organizations operating in highly secure or air-gapped environments can benefit from sovereign AI Cloud services by deploying private, on-premises AI clouds, ensuring data and model security while still leveraging the benefits of cloud-based AI infrastructure. This can be strengthened by implementing advanced techniques like <em>confidential computing</em> which is supported by <a href="https://www.nvidia.com/en-in/data-center/solutions/confidential-computing/">NVIDIA hopper architecture</a>.</li><li>Virtual machines (VMs) with GPU access for data scientists and ML engineers: very simple use case but need of the hour for every business. Their engineers need access to larger machines and powerful GPUs. Not many companies want to do this by putting a budget in CAPEX but rather utilize AI Cloud service that offers GPU-accelerated VMs tailored for data scientists and machine learning engineers, providing them with the necessary compute resources for model development, training, and experimentation.</li><li>AI Platforms: By abstracting away the complexities of model deployment and management, AI Cloud services can enable organizations to deploy large language models (LLMs) and other AI models with a simple one-click process, streamlining the deployment workflow and reducing operational overhead.</li></ol><h3>How to build AI Cloud</h3><p>As we explore the concept of building an AI Cloud, it is important to revisit the synergies between AI and cloud-native technologies. Cloud-native principles, such as containerization, microservices, and automated scaling, are highly relevant to AI workloads, enabling efficient deployment, management, and scalability of AI models and pipelines.</p><p>We would like to share our experience based on the extensive research and hands-on experience building one.</p><h3>Infrastructure Layer</h3><p>The hardware and infrastructure layer is the most critical part of AI due to its high compute and bandwidth demand, reliance, and reliability requirements. The processing power of CPU and GPUs, network bandwidth, storage performance, and how power efficient the whole system is, play an important role in the success of AI cloud. For example, as a business, you don’t want to screw up a distributed training job that is running on hundreds of expensive GPUs at the last moment or not fully utilizing its potential, it could lead to a massive waste of compute resources involved here.</p><p><strong>Compute &amp; Network</strong>: One can build the stack using bare metal servers or virtual machines (VMs) in on-prem or the cloud. Bare metal is more performance though. The compute nodes are connected with high-performance networks such as RDMA and InfiniBand for efficient data transfer and high-speed communication.</p><p><strong>GPUs</strong>: It is the heart of the AI and the major differentiator. Essentially, a pool of a variety of GPUs are required depending on the performance and their capabilities. Furthermore, additional GPU scheduling and optimization strategies through the use of orchestration layers and drivers are required. Please note that some of the features may require additional licenses from the vendor such as enabling fractional GPUs.</p><p><strong>Storage</strong>: High-speed storage solutions for handling large datasets and model artifacts. Both types of storage are essential — block storage and object storage backed by technologies such as Ceph and Minio.</p><h3>Orchestration Layer</h3><p><a href="https://www.cloudraft.io/kubernetes-consulting">Kubernetes</a> is the de facto standard for orchestration purposes. The ecosystem is broad and extensible, it is the most flexible option we suggest. In some cases, Slurm is preferred in HPC, which is a fault-tolerant and scalable cluster management and job scheduling system.</p><p>Control Plane:</p><ul><li>Custom control plane for managing and orchestrating cloud workloads and requests.</li><li>Responsible for resource allocation, scheduling, and automation.</li></ul><h3>Building Blocks</h3><p>There are various building blocks that forms the overall cloud. This is not an extensive list but some of the technologies we have used in <a href="https://www.cloudraft.io/ai-cloud-consulting">our solutions</a>.</p><ul><li><a href="https://www.cloudraft.io/blog/migrating-vms-to-kubernetes-with-kubevirt">KubeVirt</a>: It empowers virtualization atop Kubernetes, facilitating the execution of AI workloads within containerized environments with enhanced flexibility and resource utilization.</li><li>Ray or Kubeflow: These frameworks serve as powerful tools for managing distributed AI and ML workloads on Kubernetes, streamlining the orchestration process, and optimizing resource allocation.</li><li>MLflow: Essential for model management and deployment, MLflow offers robust capabilities for tracking experiments, packaging code, and deploying models seamlessly within the AI cloud environment.</li><li>Vector databases: Critical for efficient storage and retrieval of high-dimensional data, vector databases play a pivotal role in supporting AI applications that require complex data structures.</li><li>Inferencing engines: Utilizing cutting-edge solutions like <a href="https://www.cloudraft.io/blog/deploy-llms-on-kubernetes-using-openllm">vLLM</a> and Triton, we ensure optimized model serving within the AI cloud, delivering high-performance inferencing capabilities to meet diverse workload requirements.</li><li>Monitoring and logging: Employing observability and troubleshooting tools, we maintain a comprehensive view of AI workloads, enabling proactive monitoring, detection, and resolution of potential issues. See our <a href="https://www.cloudraft.io/observability-consulting">observability offering</a>.</li><li>Security and audit logging: Prioritizing data protection and compliance, we implement robust security measures and audit logging mechanisms to safeguard sensitive information and ensure regulatory compliance within the AI cloud environment.</li><li>RAG (Retrieval-Augmented Generation): Leveraging advanced techniques like RAG, we enhance the capabilities of large language models, enabling them to generate more contextually relevant and accurate responses.</li><li>Serverless inferencing: Embracing serverless architecture for on-demand and scalable model serving, we optimize resource utilization and ensure cost-effectiveness in delivering inferencing capabilities to users.</li><li><strong>GPU Optimization:</strong></li><li><a href="https://www.cloudraft.io/blog/optimizing-nvidia-gpus-in-kubernetes"><em>Partitioning</em></a><em>: By efficiently sharing GPU resources among multiple workloads, we maximize resource utilization and enhance overall system performance.</em></li><li><em>Fractional GPUs: Implementing fractional GPU allocation strategies, we optimize GPU utilization by dynamically allocating fractions of GPU resources based on workload requirements.</em></li><li>User interface (UI): Providing intuitive web-based or command-line interfaces, we empower users to seamlessly manage and interact with the AI cloud, facilitating ease of use and enhancing productivity.</li><li>Billing or Metering: offering a service without a proper metering would make the revenue officer and the consumers of the service unhappy. Have a solid usage based metering.</li><li>Multi-tenancy: design a system that allows hard isolation with quotas and limits defined for each tenant to make it more reliable and secured.</li></ul><p>Our design principle when building a cloud is to minimize friction and enable engineers and other stakeholders to move swiftly in the new era of AI.</p><h3>Reference Architecture for AI Cloud</h3><p>The following reference architecture provides a high-level overview of the key components involved in building an AI Cloud. Please note that this is a simplified representation, and we encourage readers to <a href="https://www.cloudraft.io/contact-us">schedule a meeting with our experts</a> for a more detailed discussion and tailored guidance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q5K-kB33C7KGc9U4" /></figure><p>AI Cloud Reference Architecture by <a href="http://www.cloudraft.io/">CloudRaft</a></p><h3>Key Challenges in Building AI Cloud</h3><ul><li>Complexity Across Multiple Technologies: Building an AI cloud entails integrating diverse technologies spanning cloud-native solutions, AI/ML frameworks, and infrastructure components. Negotiating this complexity demands a comprehensive understanding of each domain and the ability to orchestrate them harmoniously.</li><li>Depth and Breadth of Skills: Successfully addressing the multifaceted challenges of AI cloud construction requires profound expertise that extends across a wide spectrum of disciplines. From cloud architecture to machine learning algorithms, proficiency in various domains is indispensable for ensuring the seamless operation and performance of the AI cloud.</li><li>Learnings from Past Experiences: Leveraging the experiences and insights of those who have previously embarked on similar endeavors can significantly expedite the development process. By standing on the shoulders of those who have already navigated the intricacies of AI cloud construction, organizations can accelerate their time to market and mitigate potential pitfalls.</li><li>Optimizing GPU Utilization: Given the pivotal role of GPUs in powering AI workloads, optimizing their utilization is paramount for maximizing performance and cost efficiency. This necessitates implementing strategies and techniques to ensure that GPU resources are allocated and utilized optimally across various workloads.</li><li>Ensuring Reliability and High Availability: Building a reliable AI cloud service is imperative, as downtime can have huge repercussions, both operationally and financially. With the high cost associated with maintaining AI clusters, minimizing downtime becomes paramount to safeguarding against potential losses and maintaining operational continuity.</li></ul><p>By addressing these challenges with precision and expertise, organizations can forge ahead in their goal to establish robust and efficient AI cloud infrastructures.</p><h3>Conclusion</h3><p>The rapid advancements in AI, driven by cutting-edge research, increased computational power, and growing investments, have created a pressing need for specialized cloud infrastructure tailored to the unique requirements of AI workloads. AI Cloud services offer a compelling solution by providing scalable, on-demand access to AI-optimized hardware and software resources, enabling organizations to develop, train, and deploy AI models efficiently and cost-effectively.</p><p>By leveraging AI Cloud, organizations can overcome challenges such as lack of operational expertise, resource constraints, and supply chain issues, while benefiting from enhanced data sovereignty, access to state-of-the-art models, and the democratization of AI capabilities.</p><p><a href="https://www.cloudraft.io/ai-cloud-consulting">Building an AI Cloud</a> involves a careful orchestration of various components, including powerful hardware accelerators, high-performance networking and storage, containerization and orchestration platforms, specialized AI frameworks and tools, and robust security and monitoring measures. The reference architecture presented in this article offers a high-level blueprint for constructing an AI Cloud, highlighting the key layers and enablers required for a successful implementation.</p><p>As AI continues to transform industries and drive innovation, adopting AI Cloud services is poised to accelerate, empowering organizations to harness the full potential of these transformative technologies while mitigating the complexities and challenges associated with managing AI infrastructure in-house.</p><p><a href="https://www.cloudraft.io/contact-us">Book a free call with us to go deeper into how we are building AI Cloud. We are excited to talk to you.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d4ca915df90" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Secure Coding Best Practices]]></title>
            <link>https://anjuls.medium.com/secure-coding-best-practices-16e5204be1bc?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/16e5204be1bc</guid>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Tue, 20 Jun 2023 06:42:18 GMT</pubDate>
            <atom:updated>2023-06-20T06:42:18.127Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pn9b3eS0zwmcKw3na51A3w.png" /></figure><p>Every single day, an extensive array of fresh software vulnerabilities is unearthed by diligent security researchers and analysts. A considerable portion of these vulnerabilities emerges due to the absence of secure coding practices. Exploiting such vulnerabilities can have severe consequences, as they possess the potential to severely impair the financial or physical assets of a business, erode trust, or disrupt critical services.</p><p>For organisations reliant on their software for their operations, it becomes imperative for software developers to embrace secure coding practices. Secure coding entails a collection of practices that software developers adopt to fortify their code against cyberattacks and vulnerabilities. By adhering to coding standards that embody best practices, developers can incorporate safeguards that minimise the risks posed by vulnerabilities in their code.</p><p>In a world brimming with cyber threats, secure coding cannot be viewed as optional if a business intends to maintain its shield of protection.</p><p>This article, we will explore some anti-patterns and best practices we can include in our workflow.</p><h3>Anti-patterns</h3><p>Now, let’s briefly discuss various common mistakes or anti-patterns, categorised into insecure coding. The following are some examples:</p><ul><li>Insufficient validation of input data or processing inputs without proper encoding or sanitisation.</li><li>Constructing SQL queries by concatenating strings, making the code vulnerable to data leaks or injection attacks.</li><li>Failure to implement robust authentication, such as storing credentials in plain text without proper hashing and encryption.</li><li>Poor design of password recovery mechanisms and infrequent rotation of security keys.</li><li>Software planning and design lacking strong authorisation schemes.</li><li>Granting excessive privileges during development or troubleshooting.</li><li>Exposing sensitive information in debug logging without appropriate redaction.</li><li>Utilising third-party libraries from untrusted sources or neglecting security checks.</li><li>Unsafe handling of memory pointers or allowing pointer access beyond system boundaries.</li></ul><p>With these common mistakes in mind, let’s explore practices and tools that can guide developers towards secure coding practices.</p><h3>Secure Coding Best Practices</h3><h3>Shift left in software development lifecycle</h3><p>Historically, the conventional practice involved assigning the software security team to conduct security testing towards the conclusion of a software development project. The team would assess the application and compile a list of issues that require resolution. At this stage, the identified fixes would be prioritised, resulting in some vulnerabilities being addressed while others remained unattended. The reasons for leaving certain vulnerabilities unresolved could range from cost constraints and limited resources to pressing business priorities.</p><p>However, this traditional approach is no longer sustainable. Security considerations must now be incorporated right from the outset — the initial stages — of the software development lifecycle. Security should be taken into account during the design phase itself. Both manual and automated testing should be conducted throughout the application’s implementation as part of the Continuous Integration (CI) pipeline, ensuring that developers receive prompt feedback.</p><p>To aid in this endeavour, the utilisation of static code analysis becomes invaluable. This technique enables the scanning of code for security flaws and risks, even while developers are actively writing it within an integrated development environment (IDE). For instance, SAST tools offers the ability to analyse the code for security vulnerabilities during the development process, facilitating early identification and mitigation of potential risks.</p><h3>Input validation</h3><p>Ensuring the integrity of input data as it enters a system holds great significance. It is essential to validate the syntactic and semantic accuracy of all incoming data, considering it as untrusted. Employing checks and regular expressions aids in verifying the correctness, size, and syntax of the input.</p><p>Performing these validations on the server side is highly recommended. In the case of web applications, it involves scrutinising various components, including HTTP headers, cookies, GET and POST parameters, as well as file uploads.</p><p>Client-side validation also proves beneficial, contributing to an enhanced user experience by reducing the need for multiple network requests resulting from invalid inputs. This approach minimises back-and-forth communication and enhances efficiency.</p><h3>Parameterised queries</h3><p>During the process of storing and retrieving data, developers frequently engage with datastores. However, if they overlook the utilisation of parametrised queries, it can expose an opportunity for attackers to exploit widely accessible tools and manipulate inputs to extract sensitive information. SQL injection, a highly perilous application risk, exemplifies a common form of such attacks.</p><p>By incorporating placeholders for parameters within the query, the specified parameters are treated as data rather than being considered as part of the SQL command itself. To mitigate these vulnerabilities, it is recommended to employ prepared statements or object-relational mapping (ORM) techniques. These approaches offer effective measures to safeguard against SQL injection and related threats.</p><h3>Encoding data</h3><p>Encoding data plays a vital role in mitigating threats by transforming potentially hazardous special characters into a sanitised format. Base64 encoding serves as an exemplar of such encoding techniques, offering protection against SQL injection, cross-site scripting (XSS), and client-side injection attacks.</p><p>To enhance security, it is crucial to specify appropriate character sets, such as UTF-8, and encode data into a standardised character set before further processing. Additionally, employing canonicalisation techniques proves beneficial. For instance, simplifying characters to their basic form helps address issues such as double encoding and obfuscation attacks, thereby bolstering overall security measures.</p><h3>Implement identity and authentication controls</h3><p>To further enhance security and minimise the risk of breaches, secure coding practices emphasise the importance of verifying a user’s identity at the outset and integrating robust authentication controls into the application’s code.</p><p>Here are some recommended measures to achieve this:</p><ul><li>Employ strong authentication methods, such as multi-factor authentication, to add an additional layer of security.</li><li>Consider incorporating biometric authentication methods, such as fingerprint or facial recognition, especially in mobile applications.</li><li>Ensure secure storage of passwords. Typically, this involves hashing the password using a strong hashing function and securely storing the encrypted hash in a database.</li><li>Implement a secure password recovery mechanism to facilitate password resets while maintaining security.</li><li>Enable session timeouts and inactivity periods to automatically terminate idle sessions.</li><li>For sensitive operations like modifying account information, enforce re-authentication to validate the user’s identity.</li><li>Conduct regular audits of authentication transactions to detect any suspicious activities and maintain a vigilant stance against potential threats.</li></ul><h3>Implement access controls</h3><p>Incorporating a well-thought-out authorisation strategy during the initial stages of application development can greatly enhance the overall security posture. Authorisation entails determining the specific resources that an authenticated user can or cannot access.</p><p>Consider the following guidelines to strengthen the authorisation framework:</p><ul><li>Establish a sequential flow of authentication followed by authorisation. Implement a mechanism where all requests undergo access control checks.</li><li>Adhere to the principle of least privilege, initially denying access to any resource that has not been explicitly configured for access control.</li><li>Enforce time-based limitations on user or system component actions by implementing expiration times, thereby ensuring that actions have defined timeframes for execution.</li></ul><p>By following these practices, developers can create a robust and effective authorisation system that bolsters the overall security of the application.</p><h3>Protect sensitive data</h3><p>In order to comply with legal and regulatory obligations, it is the responsibility of businesses to safeguard customer data. This sensitive data encompasses various categories, including:</p><ul><li>Personally identifiable information (PII)</li><li>Financial transactions</li><li>Health records</li><li>Web browser data</li><li>Mobile data etc</li></ul><p>To prevent data leakage, it is crucial to employ robust encryption methods for both data at rest and data in transit. Consider the following practices to enhance data protection:</p><ul><li>Utilise a well-established, peer-reviewed cryptographic library and functions that have been vetted and approved by your security team.</li><li>Avoid storing encryption keys alongside the encrypted data to prevent unauthorised access.</li><li>Refrain from storing confidential or sensitive data in memory, temporary locations, or log files during processing.</li><li>Implement redaction technique in log forwarders to remove sensitive information.</li><li>Implement mandatory re-authentication when accessing sensitive data within the application.</li></ul><h3>Implement logging and intrusion detection</h3><p>Even the most meticulously designed system can be susceptible to exploitation by attackers. Therefore, it is advisable to incorporate a monitoring system that can detect and identify unusual events. It is crucial to ensure that sufficient information is logged concerning authentication, authorisation, and resource access events. This logging should include details such as timestamps, the origin of access requests, IP addresses, and information pertaining to the requested resource. It is important to store this information in a secure and protected log. Typically, these logs are transmitted in real time to a centralised system where they are analysed for any anomalies. Prior to logging, apply encoding techniques to the untrusted data to safeguard against log injection attacks.</p><p>In the event of a security breach, it is essential to have a well-documented playbook in place to promptly terminate system access, mitigating the risk of further data leakage. By following these practices, organisations can enhance their ability to detect and respond to potential intrusions, minimising the impact of security incidents.</p><h3>Leverage security frameworks and libraries</h3><p>Avoid unnecessary duplication of effort. Instead, leverage established security frameworks and libraries that have been proven effective. When incorporating such components into your project, ensure they are sourced from reliable and trusted third-party repositories. It is important to regularly assess these libraries for any vulnerabilities or weaknesses and proactively keep them up to date.</p><p>By adopting this approach, you can benefit from the expertise and experience embedded in these established security solutions, saving valuable time and effort while maintaining a strong security posture.</p><h3>Monitor error and exception handling</h3><p>In line with the best practices of logging, it is advisable to adopt a centralised approach for handling and monitoring errors and exceptions with tools like Sentry. Effective management of errors and exceptions is crucial as mishandling them can inadvertently expose valuable information to potential attackers, enabling them to gain insights into your application and platform design.</p><p>Consider the following measures to strengthen error and exception handling:</p><ul><li>Avoid logging sensitive information within error messages to prevent inadvertent disclosure.</li><li>Regularly conduct code reviews to identify and address any weaknesses or vulnerabilities in the error handling implementation.</li><li>Utilise negative testing techniques, such as exploratory and penetration testing, fuzzing, and fault injection, to actively identify and rectify potential issues related to error handling.</li></ul><p>By implementing these practices, you can ensure that error and exception handling is performed securely and with minimal risk of exposing sensitive information to potential attackers.</p><h3>Benefits of implementing secure coding practices</h3><p>At this point, the advantages of embracing secure coding practices should be evident:</p><ul><li>Incorporating automated checks and code analysis during the development process enhances developer productivity by promptly providing feedback to improve code security. This leads to quicker time-to-market and higher-quality code.</li><li>Cost optimisation within the software development lifecycle is achieved by minimising bugs at the early stages.</li><li>Static application security testing (SAST) tools offer developers of all skill levels guardrails, AppSec governance, and valuable insights through IDE plugins. These tools equip developers with the necessary knowledge and resources to bolster application security.</li></ul><h3>Conclusion</h3><p>Throughout our examination of coding flaws that can result in vulnerabilities, we have also explored best practices to enhance the security stance of software. However, in the context of large-scale projects, it can be daunting to implement these practices while ensuring proper governance.</p><p>In the realm of extensive projects, the following considerations can help navigate these challenges effectively:</p><ul><li>Establish clear governance frameworks that outline security requirements, procedures, and responsibilities.</li><li>Develop comprehensive guidelines and standards that align with secure coding practices and provide actionable steps for implementation.</li><li>Foster collaboration and communication among development teams, security experts, and stakeholders to ensure a shared understanding of security goals and the necessary measures to achieve them.</li><li>Prioritise the implementation of security measures by identifying high-risk areas and focusing resources accordingly.</li><li>Regularly assess and review the security posture of the software throughout the development lifecycle, enabling continuous improvement and adjustments as needed.</li></ul><p>By adopting these approaches, the process of implementing secure coding practices within large projects becomes more manageable and ensures that proper governance is in place to safeguard against vulnerabilities effectively.</p><p>It is advisable to create and automate workflows using SAST tools and integrate them into CI to enforce the best practices. Feel free to schedule a non-obligatory <a href="https://cloudraft.io/contact-us">call with us</a> to discuss DevSecOps strategy and we can help you improve your current practice.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16e5204be1bc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Taking AI/ML Ideas to Production]]></title>
            <link>https://faun.pub/taking-ai-ml-ideas-to-production-48244f15c269?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/48244f15c269</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[model]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Tue, 04 Apr 2023 17:22:15 GMT</pubDate>
            <atom:updated>2023-04-05T06:40:52.870Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xJhtviA3ai7qyE9GghzDsA.png" /></figure><p>The integration of AI and ML in products has become a trend in recent years. Companies are trying to incorporate these technologies into their products to improve their efficiency and performance. And this year particularly, with the boom of ChatGPT, almost every company is trying to introduce a feature in this domain. One of the main benefits of AI and ML is their ability to learn and adapt. They can analyze data and use it to improve their performance over time. This means that products that incorporate these technologies can become smarter and more efficient over time.</p><p>Let’s understand now, how companies are taking their ideas to production. Usually, they start with hiring a few data scientists who will figure out what models to create to solve the problem, fine-tune them, and handover to MLOps or DevOps engineers to deploy. Your DevOps engineers may or may not know, how to efficiently take these models to production. That’s where you need specialized skills such as Machine learning engineers and MLOps who understand how to manage the whole process of CI/CD/CT pipeline efficiently.</p><h3>Maturity of Deployment Strategy</h3><p>Many engineers will start with packaging the model and APIs in a popular Python framework like Flask or FastAPI, in a container and deploy on Docker or Kubernetes. This works well for the lab type of environments but is not meant for production use cases.</p><p>More mature companies come up with their tooling to orchestrate and deploy the service. I think they are well set but their system is not aligned with the ecosystem and requires a lot of effort in managing and maintaining the system over time.</p><p>Lastly, you are at the rightmost side where you deploy a specialized machine learning platform such as Kubeflow, Ray, ClearML, etc. which provides end-to-end tools to manage the lifecycle of ML service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/0*8Twm_rvDPPBhMShJ" /></figure><p><em>Credits: Model Serving at the Edge Made Easier — Paul Van Eck &amp; Animesh Singh, IBM, </em><a href="https://www.youtube.com/watch?v=0BlK7PaLCFM"><em>Youtube</em></a></p><h3>Where to Start?</h3><p>If you are new to MLOps, it is not so easy to understand what exactly you need in your stack. To simplify this, MLOps community has shared a template that can help you to do some self-assessment, and navigate the large AI/ML platform ecosystem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RW4S-DU6QN1Dmxpg" /></figure><p>Not all the components are required in the stack but you can put your requirements for each component and identify the tool that works for you. Essentially, you need a way to bring data to the platform with version control, run and record experimentations, an ML pipeline to automatically run the code that your data scientist is framed in the notebook, and a model registry where you will store the models and their lineage, followed by model serving and monitoring the performance of the inference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AwitcKZY0qpHI4BM" /></figure><p><em>Lifecycle of AI/ML Project</em></p><h3>Finding the Right Tool for the Job</h3><p>As I stated earlier, the ecosystem is thriving and there are hundreds of tools and frameworks coming up to manage a subset or the full lifecycle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q-SttfFmgQISmUfz" /></figure><p><em>Source: nepture.ai </em><a href="https://neptune.ai/wp-content/uploads/2022/10/MLOps-tools-landscape-horizontal-upd.png"><em>image</em></a></p><p>Neptune.ai has compiled the above stack which I find quite useful to understand how these offerings are fitting together. Here are some choices that you can explore.</p><p>Here are some choices that you can explore.</p><ul><li>Cloud offering: If you are in any major public cloud, you will get access to the services like Vertex AI in Google Cloud, Sagemaker in AWS, and Azure ML in Microsoft Azure. They have done a pretty good job making useful end-to-end lifecycle management. Some of them are more mature than others obviously and some of them are not very cost-effective. For example, I don’t find their model serving options not very efficient. They don’t allow you to use fractional GPUs or run a copy of a full GPU machine during deployment rollout.</li><li>Build your stack on Kubernetes: If you are brave and want to take some open-source framework like Kubeflow or Ray, you can create your stack. These frameworks are mostly complete when combined with MLFlow.</li><li>Commercial Products: Lastly, there are players like truefoundry, weights and biases, ClearML, etc that provide the full solution with minimal operational overhead. ClearML is also available as an open-source self-hosted version but then you back to point 2, building your stack.</li></ul><p>There are many more tools and products available, but I don’t want to focus on them instead I want to give a rough sketch of the stack. Whatever you select, the de facto industry standard to host these MLOps stacks is Kubernetes, sometimes cloud provides management for you to simplify the operations and sometimes leaves it to you to run on your clusters.</p><h3>Challenges</h3><ol><li>Most often, we overlook data privacy and security and only realize this when we are hacked or need to get some compliance.</li><li>Lack of skilled resources who understand the ecosystem well.</li><li>Sparsely spread tooling which is not production ready. For example, I have found many popular tools in the MLOps stack lacking RBAC and authentication.</li><li>A wrong or lack of understanding of the right stack leads to inefficient processes.</li><li>Inefficient use of computing such as GPU can blow your cost. We often overlook cost initially and realize only later when we are hit by a bill shock.</li><li>Inefficient underlying platform — this could be your Kubernetes cluster which is not efficiently handling computing, has unreliable cluster design, etc.</li></ol><h3>Need Expert Help</h3><p>I am sure you might have been overwhelmed by the CNCF landscape, and how complex it has become over time. If you are stuck finding a proper MLOps stack, <a href="https://www.cloudraft.io/contact-us">book some time</a> to chat about your problem, we will be more than happy to help you.</p><p>We at CloudRaft help businesses grow and solve complex problems by leveraging cloud-native technologies and modern platform engineering practices. We are building the MLOps stack for our clients and learning about the evolving ecosystem. Do ping us (<a href="/service/mailto:info@cloudraft.io">info@cloudraft.io</a>) if you need help in these areas. If you like this article, don’t forget to share it with your friends and colleagues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*kBpXaD5BsvdQfO_O.png" /></figure><h4>👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀<a href="http://from.faun.to/r/8zxxd">Join FAUN Developer Community &amp; Get Similar Stories in your Inbox Each Week</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48244f15c269" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/taking-ai-ml-ideas-to-production-48244f15c269">Taking AI/ML Ideas to Production</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing gRPC-Web with Emissary-ingress]]></title>
            <link>https://medium.com/ambassador-api-gateway/implementing-grpc-web-with-emissary-ingress-22aa0d86aac?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/22aa0d86aac</guid>
            <category><![CDATA[envoy]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[emissary]]></category>
            <category><![CDATA[grpc]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Tue, 23 Aug 2022 14:18:27 GMT</pubDate>
            <atom:updated>2022-09-08T11:34:33.940Z</atom:updated>
            <content:encoded><![CDATA[<h4>Guest Writer</h4><p><a href="https://grpc.io/">gRPC</a> is a high-performance and low-latency Remote Procedure Call framework created by Google. <a href="https://github.com/grpc/grpc-web">gRPC-Web</a> was built on the foundation of the gRPC protocol to provide a high-performance architecture to implement services natively in the browser, using HTTP2 and gRPC to have a bi-directional stream and efficient multiplexing.</p><p><a href="https://www.getambassador.io/learn/envoy-proxy/">Envoy</a> proxy is an important element when working with gRPC or gRPC-web because it helps to translate the gRPC Web calls over HTTP, and the impact of this on performance is phenomenal.</p><p>In this article, I’ll show you how to implement gRPC-Web with <a href="https://www.getambassador.io/products/api-gateway/">Emissary-ingress</a> — an Envoy-based API gateway and Ingress controller.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nIvDe0-sqYjmNAPKcZlRAQ.jpeg" /></figure><h4>gRPC in Kubernetes using Emissary-ingress</h4><p>To understand how gRPC-Web works, we will set up a <strong>Greeter service</strong> demo that uses gRPC protocol to respond to the Hello request. This service will be deployed on a Kubernetes cluster and exposed using the Emissary-ingress API gateway.</p><p>I’ll be using Minikube to create a Kubernetes cluster, so if you don’t have a Kubernetes cluster yet, you can create one by following this <a href="https://minikube.sigs.k8s.io/docs/start/">guide</a>. After setting up the Kubernetes cluster on your computer using Minikube, start the cluster using the minikube start command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e-vLz95um5O9k54ticUtsw.png" /><figcaption>Minikube</figcaption></figure><p>The next thing we are going to do is install Emissary-ingress and it&#39;s custom CRDs (Mappings, Listeners, Hosts) by running the command below on our minikube cluster.</p><pre># Add the Repo:<br>helm repo add datawire <a href="https://app.getambassador.io/">https://app.getambassador.io</a><br>helm repo update<br> <br># Create Namespace and Install:<br>kubectl create namespace emissary &amp;&amp; \<br>kubectl apply -f <a href="https://app.getambassador.io/yaml/emissary/2.2.2/emissary-crds.yaml">https://app.getambassador.io/yaml/emissary/2.2.2/emissary-crds.yaml</a><br>kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system<br>helm install emissary-ingress --namespace emissary datawire/emissary-ingress &amp;&amp; \<br>kubectl -n emissary wait --for condition=available --timeout=90s deploy -lapp.kubernetes.io/instance=emissary-ingress</pre><p>To route the traffic from the edge, we will create an HTTP listener on port 8080 by applying the YAML configuration below on our Kubernetes cluster.</p><pre>kubectl apply -f - &lt;&lt;EOF<br>---<br>apiVersion: getambassador.io/v3alpha1<br>kind: Listener<br>metadata:<br>  name: emissary-ingress-listener-8080<br>  namespace: emissary<br>spec:<br>  port: 8080<br>  protocol: HTTP<br>  securityModel: XFP<br>  hostBinding:<br>    namespace:<br>      from: ALL<br>EOF</pre><h4>Hands-on with Emissary-ingress and gRPC-web</h4><p>The Greeter service structure will be defined by a <em>greeter.proto</em> file using Protocol Buffers — a language-neutral, platform-neutral extensible mechanism for serializing structured data. So create a file named greeter.proto and add the contents below to it.</p><pre>syntax = &quot;proto3&quot;;<br>package greeter;</pre><pre>service Greeter {<br>  rpc Hello (HelloRequest) returns (HelloReply) {}<br>}</pre><pre>message HelloRequest {<br>  string name = 1;<br>}</pre><pre>message HelloReply {<br>  string message = 1;<br>}</pre><p>As you can see, the greeter.proto file has a gRPC service definition for the Greeter service called Hello, which will access a HelloRequest message and return a HelloReply message.</p><p>I’ve also added a diagrammatical representation of the flow of the greeter.proto service to give you a better perspective of how it works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/0*013qmv2F3d0j_e6i.png" /></figure><h4>Generate the Stubs for the Service</h4><p>Compiling the protobuf file with common.js style will generate the stubs needed for the gRPC service implementation. You can generate them using the protocol CLI.</p><pre>protoc greeter.proto --js_out=import_style=commonjs:./ --grpc-web_out=import_style=commonjs,mode=grpcwebtext:./</pre><p>The compilation will generate two stubs named greeter_pb.js and greeter_grpc_web_pb.js.</p><h4>Implementing the gRPC Greeter Service</h4><p>Now we can implement the gRPC service, which will have a function doHello that returns the message. First, create a file named server.js and add the contents below to it.</p><pre>const PORT = 9090;<br>const PROTO_PATH = `${__dirname}/greeter.proto`;<br>const grpc = require(&quot;grpc&quot;);<br>const protoLoader = require(&quot;<a href="http://twitter.com/grpc/proto-loader">@grpc/proto-loader</a>&quot;);<br>const packageDefinition = protoLoader.loadSync(PROTO_PATH, {<br>  keepCase: true,<br>  longs: String,<br>  enums: String,<br>  defaults: true,<br>  oneofs: true<br>});<br>const protoDescriptor = grpc.loadPackageDefinition(packageDefinition);<br>function doHello(call, callback) {<br>  callback(null, {<br>    message: &#39;Hello! &#39; + call.request.name<br>  });<br>}</pre><pre>function getGrpcServer() {<br>  const server = new grpc.Server();<br>  server.addService(protoDescriptor.greeter.Greeter.service, {<br>    Hello: doHello<br>  });<br>  return server;<br>}</pre><pre>if (require.main === module) {<br>  var server = getGrpcServer();<br>  server.bind(`0.0.0.0:${PORT}`, grpc.ServerCredentials.createInsecure());<br>  server.start();<br>  console.log(`Server started on port ${PORT}`);<br>}</pre><h4>Deploy the Greeter Service</h4><p>Now, let’s containerize this application by creating a docker image. To do this, create a file named <strong>Dockerfile </strong>in the same directory of the server.js file and paste the content below into it.</p><pre>FROM node:14.19<br>WORKDIR /app<br>COPY [&quot;package.json&quot;, &quot;server.js&quot;, &quot;./&quot;]<br>RUN npm install --no-cache --production<br>COPY . .<br>EXPOSE 9090<br>CMD [ &quot;node&quot;, &quot;server.js&quot; ]</pre><p>After creating the <strong>Dockerfile</strong>, go back to your Kubernetes cluster and run the command below to create a running container based on the docker image you created in the previous step.</p><pre>docker build . -t grpc-server:1.0 </pre><p>After doing this, you can then push it to DockerHub or Minikube. This is optional, as you can still go ahead with the article without completing this step.</p><pre># optional, push the image to minikube<br>minikube image load grpc-server:1.0</pre><p>The next thing we are going to do is deploy the <strong>Greeter service</strong>. To do this, create a file named deploy.yaml and paste the content below into it.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: greeter<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: greeter<br>  template:<br>    metadata:<br>      labels:<br>        app: greeter<br>    spec:<br>      containers:<br>      - name: greeter<br>        image: grpc-server:1.0<br>        ports:<br>        - containerPort: 9090<br>          name: http<br>          protocol: TCP<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: greeter<br>spec:<br>  selector:<br>    app: greeter<br>  type: ClusterIP<br>  ports:<br>  - name: http<br>    port: 9090<br>    targetPort: 9090<br>---</pre><p>Then go to your Kubernetes cluster and run this command kubectl apply -f deploy.yaml to apply/create the Kubernetes resources inside your cluster.</p><h4>Configure Emissary-ingress resources to expose the Greeter Service</h4><p>Let’s create Module, Mapping, and Host resources to expose the Greeter service. After reading the content of each section, copy the YAML content into a file name of your choice and apply them to your Kubernetes cluster using the kubectl apply -f filename.yaml command.</p><h4><strong>Module</strong></h4><p>A module defines the system-wide configuration of Emissary-Ingress. To use the ambassador module, it must be named as <strong>ambassador</strong>. There are two items in the ambassador module named module — enable_grpc_web and enable_grpc_http11_bridge. Both must be set to true to enable the gRPC Web bridge.</p><p>This bridge translates the HTTP/1.1 calls with the header Content-Type: application/grpc to gRPC calls. You can find more details about this in the official <a href="https://www.getambassador.io/docs/emissary/latest/topics/running/ambassador/">documentation</a>.</p><pre>apiVersion: getambassador.io/v2<br>kind:  Module<br>metadata:<br>  name:  ambassador<br>spec:<br># Use ambassador_id only if you are using multiple instances of Emissary-ingress in the same cluster.<br># See below for more information.<br>  config:<br>  # Use the items below for config fields<br>    use_proxy_proto: true<br>    enable_grpc_web: true<br>    enable_grpc_http11_bridge: true<br>    diagnostics:<br>      enabled: true</pre><h4>Mapping</h4><p>The ambassador Mapping resource can be used to map a resource to a service. Here the /greeter.Greeter/would route to greeter service on port 9090 using gRPC. We have defined other things, particularly headers, to enable it for gRPC Web. More details in the <a href="https://www.getambassador.io/docs/emissary/latest/topics/using/intro-mappings/">reference</a>.</p><pre>apiVersion: getambassador.io/v3alpha1<br>kind: Mapping<br>metadata:<br>  name: grpc-greeter<br>spec:<br>  hostname: &quot;*&quot;<br>  grpc: True<br>  prefix: /greeter.Greeter/<br>  rewrite: /greeter.Greeter/<br>  service: greeter.default:9090<br>  timeout_ms: 4000<br>  idle_timeout_ms: 500000<br>  connect_timeout_ms: 2000<br>  bypass_auth: true<br>  dns_type: logical_dns<br>  cors:<br>    origins:<br>    - &quot;*&quot;<br>    methods: <br>    - POST<br>    - GET<br>    - OPTIONS<br>    - PUT<br>    - DELETE<br>    headers: <br>    - keep-alive<br>    - user-agent<br>    - cache-control<br>    - content-type<br>    - content-transfer-encoding<br>    - custom-header-1<br>    - x-accept-content-transfer-encoding<br>    - x-accept-response-streaming<br>    - x-user-agent<br>    - x-grpc-web<br>    - grpc-timeout<br>    exposed_headers: <br>    - custom-header-1<br>    - grpc-status<br>    - grpc-message</pre><h4>Host</h4><p>A Host resource describes how you want Emissary-Ingress to look externally. The YAML configuration file of a Host resource usually contains a hostname, how to handle insecure connections, the TLS configuration, and the Mappings linked to the Host.</p><pre>apiVersion: getambassador.io/v3alpha1<br>kind: Host<br>metadata:<br>  name: example-host<br>spec:<br>  hostname: localhost<br>  acmeProvider:<br>    authority: none<br>  requestPolicy:<br>    insecure:<br>      action: Route</pre><h4>Deploying Client Side Stubs</h4><p>At this point, you should have a gRPC server running in the Kubernetes cluster and exposed via Emissary-Ingress. We will now deploy the client-side stubs we created earlier using the Protobuf compiler in common.js format.</p><p>The next thing we need to do is import the stubs and call the Greeter service. To do this, create a file namedclient.js and add the content below.</p><pre>const  { HelloRequest, HelloReply} = require(&#39;./greeter_pb.js&#39;);<br>const  { GreeterClient } = require(&#39;./greeter_grpc_web_pb.js&#39;);<br>const client = new GreeterClient(&#39;<a href="http://localhost&/#39;">http://localhost&#39;</a>);<br>// console.log(client)<br>var request = new HelloRequest();<br>request.setName(&#39;Ambassador!&#39;);<br>client.hello(request, {}, (err, response) =&gt; {<br>  console.log(response.getMessage());<br>});</pre><p>I have created a tunnel using the<a href="https://kubernetes.io/docs/tasks/access-application-cluster/port-forward-access-application-cluster/"> kubectl port-forward</a> command to reach the Emissary-Ingress service running inside the Minikube cluster on local port 80.</p><p>Now, let’s package our client’s code and generate a distribution to run in the browser using a Webpack. Run the below commands in your terminal to generate the distribution.</p><pre>npm install <br>npx webpack client.js</pre><p>This will generate ./dist/main.js distribution, which we will call on the web page. Next, create a file named index.html and add the content below to return a sample web page.</p><pre>&lt;!DOCTYPE html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br>  &lt;head&gt;<br>    &lt;meta charset=&quot;UTF-8&quot; /&gt;<br>    &lt;title&gt;gRPC-Web Example&lt;/title&gt;<br>    &lt;script src=&quot;./dist/main.js&quot;&gt;&lt;/script&gt;<br>  &lt;/head&gt;<br>  &lt;body&gt;<br>    &lt;p&gt;open developer console of the browser.&lt;/p&gt;<br>  &lt;/body&gt;<br>&lt;/html&gt;</pre><p>At this point, we should host the script and static page in some webserver. We will use python’s HTTP server module in this demo to start the webserver at port 8081.</p><pre>python3 -m http.server 8081 &amp;</pre><p>Now open the browser and point it to <a href="http://localhost:8081.">http://localhost:8081</a>. This will load the page below and call the main.js client-side javascript. You can see the response in the console coming from the gRPC service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UbirDFIeGHSO1n4k.png" /></figure><h4>Conclusion</h4><p>To recap, we created a Greeter gRPC service that greets you — it uses gRPC end-to-end, from the browser to the Kubernetes service via the Envoy proxy-based API gateway, Emissary-Ingress. Here’s a <a href="https://github.com/anjuls/grpc-web-with-emissary-ingress">GitHub repository</a> containing everything we’ve done so far.</p><p>Thanks for reading. If you have any questions or concerns, do well to leave a comment on this article, and I’ll respond to it as soon as I can.</p><h4>Are you looking for the right tool to route traffic to your Kubernetes services?</h4><p>Routing traffic into your Kubernetes cluster requires modern traffic management. And that’s why we built Ambassador Edge Stack (the commercial version of Emissary Ingress) to contain a modern Kubernetes ingress controller that supports a broad range of protocols, including HTTP/3, gRPC, gRPC-Web, and TLS termination.</p><p><a href="https://www.getambassador.io/docs/edge-stack/latest/tutorials/getting-started/?utm_source=AGW&amp;utm_medium=blog&amp;utm_campaign=AES">Try Ambassador Edge Stack today</a> or learn more about <a href="https://www.getambassador.io/products/edge-stack/api-gateway/?utm_source=AGW&amp;utm_medium=blog&amp;utm_campaign=AES">Ambassador Edge Stack</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QLpJpn7zFVZC7ieD" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22aa0d86aac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ambassador-api-gateway/implementing-grpc-web-with-emissary-ingress-22aa0d86aac">Implementing gRPC-Web with Emissary-ingress</a> was originally published in <a href="https://medium.com/ambassador-api-gateway">Ambassador Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top Resources to learn about Cloud Native Technologies]]></title>
            <link>https://faun.pub/top-resources-to-learn-about-cloud-native-technologies-eb0bd0804456?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb0bd0804456</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[newsletter]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Sat, 02 Jul 2022 00:30:37 GMT</pubDate>
            <atom:updated>2022-07-02T00:30:37.481Z</atom:updated>
            <content:encoded><![CDATA[<p>As per the CNCF survey of 2021, about 96% of organizations are currently using or evaluating Kubernetes. The ecosystem is growing at a high pace and it is increasingly difficult to consume the news, innovations, and updates from 1000s of projects. That is where curated resources are helpful to cut through the ocean of information.</p><p>In this article, I try to consolidate the best newsletters and podcasts to subscribe to be on top of the news and learning resources.</p><blockquote>If I have seen further than others, it is by standing upon the shoulders of giants. — Sir Issac Newton</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*mXlXj8jADwUOfrZgoH9kxA.png" /></figure><h3>Newsletters</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PrEpGncxjYOn0EK5kq75fw.png" /></figure><p><a href="https://anjulsahu.substack.com/"><strong>Cloud Native Weekly</strong></a><strong> </strong>— It is a weekly newsletter covering top curated articles in cloud-native, security, and other resources.</p><p><a href="https://www.cncf.io/kubeweekly/"><strong>kubeweekly</strong></a> — Official newsletter of Linux Foundation which is published every week and has the best-curated content from the community and CNCF partners.</p><p><a href="https://thenewstack.io/"><strong>The New Stack</strong></a> — One of the top publications covering cloud computing, cloud-native blog posts, and articles. You can read the archives <a href="https://thenewstack.io/newsletter-archive">here</a>.</p><p><a href="http://lwkd.info/"><strong>Last Week in Kubernetes Development</strong></a><strong> </strong>— This feed focuses on the latest features and important issues or changes in the Kubernetes development.</p><p><a href="https://eks.news/"><strong>EKS News</strong></a><strong> </strong>— It not only covers the news and latest updates in the EKS but also picks up the news from the ecosystem.</p><p><a href="https://o11y.news/"><strong>o11y News</strong></a><strong> </strong>— It covers observability really very well. Thanks to Michael Hausenblaus for taking the time to share his curated articles.</p><p><a href="https://devopsish.com/"><strong>DevOps’ish</strong></a><strong> </strong>— Curated articles by Chris Short, covering open-source, cloud-native, and DevOps.</p><h3>Podcasts</h3><p><a href="https://kubernetespodcast.com/">Kubernetes Podcast from Google</a> — for many years, this has been my favorite place to go when doing a commute. It covers the news and has discussions with one of the project contributors.</p><p><a href="https://thepodlets.io/">Podlets</a> by VMWare — another top podcast to listen to.</p><h3>Youtube Playlists</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPL7bmigfV0EqQzxcNpmcdTJ9eFRPBe-iZa&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fplaylist%3Flist%3DPL7bmigfV0EqQzxcNpmcdTJ9eFRPBe-iZa&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fil8EBSnnyYE%2Fhqdefault.jpg%3Fsqp%3D-oaymwEWCKgBEF5IWvKriqkDCQgBFQAAiEIYAQ%3D%3D%26rs%3DAOn4CLC3j-6JxWwRCl7NRrVPMko76rNrXA%26days_since_epoch%3D19174&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="853" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cecbbc4bd82544aa7af87998c7c974da/href">https://medium.com/media/cecbbc4bd82544aa7af87998c7c974da/href</a></iframe><p>Joe Beda and his friends started the TGI Kubernetes where they explore the news and one of the projects in their live discussion.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLDg_GiBbAx-mY3VFLPbLHcxo6wUjejAOC&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fplaylist%3Flist%3DPLDg_GiBbAx-mY3VFLPbLHcxo6wUjejAOC&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FzdbCvGSdFiQ%2Fhqdefault.jpg%3Fsqp%3D-oaymwEWCKgBEF5IWvKriqkDCQgBFQAAiEIYAQ%3D%3D%26rs%3DAOn4CLCuk7AHzMITXKu0Fk7ZbAwcfEfENQ%26days_since_epoch%3D19174&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="853" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/1f3b251ae9344d55bb46f5ff8788c249/href">https://medium.com/media/1f3b251ae9344d55bb46f5ff8788c249/href</a></iframe><p>eCHO livestream by Cilium is focussing on eBPF and other projects in their frequent talks.</p><p><a href="https://www.youtube.com/c/DevOpsToolkit">DevOps Toolkit</a></p><p>DevOps Toolkit channel on Youtube by Viktor Farcic talks about the tools and processes and greater detail. You will surely enjoy Viktor’s way of teaching.</p><p>That’s it for this article. I hope you will enjoy learning from the experts. To get more updates, please follow me on Medium (<a href="https://medium.com/u/808537596ad5">Anjul Sahu</a>) and <a href="http://twitter.com/anjuls">Twitter</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*9c48LzUS4Skdt0R5.png" /></figure><h4>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</h4><h4>🚀Developers: Learn and grow by keeping up with what matters, <a href="https://faun.to/8zxxd">JOIN FAUN.</a></h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb0bd0804456" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/top-resources-to-learn-about-cloud-native-technologies-eb0bd0804456">Top Resources to learn about Cloud Native Technologies</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top 10 Container Security Best Practices]]></title>
            <link>https://faun.pub/top-10-container-security-best-practices-317d8070b9a9?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/317d8070b9a9</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Thu, 10 Mar 2022 08:09:28 GMT</pubDate>
            <atom:updated>2022-03-24T08:58:36.752Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ljgUgSGNCAQm0hFG" /><figcaption>Photo by <a href="https://unsplash.com/@cbarbalis?utm_source=medium&amp;utm_medium=referral">Chris Barbalis</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>A cybersecurity incident can cause severe damage to the reputation of the organization and competitive disadvantage in the market, the imposition of penalties, and unwanted legal issues by end-users. On average, the cost of each data breach is USD 3.92 million as per this <a href="https://www.ibm.com/security/data-breach">IBM report</a>. The biggest challenges providing security in organizations are:</p><ul><li>Lack of skills and training in security tools and practices</li><li>Lack of visibility and vulnerabilities</li><li>Continuous monitoring of the current state of security</li></ul><p>In the recent survey by PaloAlto Networks, <a href="https://www.paloaltonetworks.com/state-of-cloud-native-security">State of Cloud Security report</a>, it was discovered that 94% of organizations use one or more cloud platforms and around 45% of their compute is on containers or CaaS. The dominance of containers is increasing and thus the security threats. The top issues identified as being a threat in these reports are:</p><ul><li>Data exposure and malware</li><li>Application vulnerabilities</li><li>Weak or broken authentication</li><li>Misconfigurations</li><li>Incorrect or over-permission access</li><li>Insider threats</li><li>Credential leakage</li><li>Insecure Endpoints</li></ul><p>In this article, we will go through some container security best practices, we can follow and implement to reduce the security risks in containerized workloads.</p><h3>Top 10 security best practices to do to secure the application containers</h3><h3>1. Source base image from trusted repositories</h3><p>When we create a container image, we often rely on the seed image sourced from popular private or public registries. Be aware that in the supply chain of the image production, someone can penetrate and drop malicious code which could open the doors to attackers. Just to give an example of this, in 2018, some <a href="https://www.wired.com/story/british-airways-hack-details/">hacker targetted British Airways web application with malicious javascript code</a> by attacking their software supply chain. A couple of years back, Docker <a href="https://threatpost.com/malicious-docker-containers-earn-crypto-miners-90000/132816/">identified few images on Docker Hub</a> which were having Cryptominers installed in the Image.</p><p>Below are some tips:</p><ul><li>When creating the container image, please use hardened base image sources from a well-known trusted publisher.</li><li>Pick those images which are published frequently with the latest security fixes and patches.</li><li>Use signed and labeled images (sign with <a href="https://www.infracloud.io/blogs/enforcing-image-trust-docker-containers-notary/">Notary</a> or similar tools) and verify the authenticity of the image during the pull to stop man-in-the-middle attacks.</li></ul><h3>2. Install verified packages</h3><p>As much as the sourcing of the base image needs to be from trusted sources, the packages installed on top of the base also need to be from verified and trusted sources for the same reason.</p><h3>3. Minimize attack surface in the image</h3><p>What I mean by surface area is the number of packages and libraries installed in the image. Common sense is that if the number of objects is less, the chances of having a vulnerability are also reduced. Keep the image size minimal, satisfying the application runtime requirements. Preferably, have only a single Application in one application container.</p><ul><li>Remove unnecessary tools and software like package managers (eg. yum, apt), network tools and clients, shells from the image, netcat (can be used to create reverse shell).</li><li>Use the multi-stage Dockerfiles to remove software build components out of production images.</li><li>Do not expose unnecessary network ports, sockets or run unwanted services (e.g. SSH daemon) in the container to reduce threats.</li><li>Choose alpine images or scratch images or container optimized OS as compared to full-blown OS images for the base image.</li></ul><h3>4. Do not bake secrets in the image</h3><p>All the secrets should be kept out of the image and Dockerfile. The secrets include SSL certificates, passwords, tokens, API keys, etc should be kept outside and should be securely mounted through the container orchestration engine or external secret manager. Tools like Hashicorp Vault, Cloud provided secret management services like AWS Secrets Manager, Kubernetes secrets, <a href="https://www.docker.com/blog/docker-secrets-management/">Docker secrets management</a>, CyberArk, etc. can improve the security posture.</p><h3>5. Use of secure private or public registries</h3><p>Often the enterprises have their own base images with proprietary software and libraries which they don’t want to distribute in public. Ensure the image is hosted on a secure and trusted registry to prevent unauthorized access. Use a TLS certificate with trusted Root CA, and implement strong authentication to prevent MITM attack.</p><h3>6. Do not use privileged or root user to run the application in a container</h3><p>This is the most common misconfiguration in the containerized workload. With principles of least privileges in mind, create an application user and use it to run the application process inside the container. Why not root? The reason is that a process running in a container is similar to the process running on the host operating system except for the fact that it has additional metadata to identify that it is part of a container. With UID and GID of root user in a container, you can access and modify the files written by root on the host machine.</p><p>Note — If you don’t define any USER in the Dockerfile, it generally means that the Container will be running with root user.</p><h3>7. Implement image vulnerability scanning in CI/CD</h3><p>When designing CI/CD for the container build and delivery, include image scanning solution to identify vulnerabilities (CVEs) and do not deploy exploitable images without remediation. Tools like Clair, Synk, Anchore, AquaSec, Twistlock can be used. Some of the container registries like AWS ECR, Quay.io are equipped with scanning solutions — do use them.</p><h3>8. Enable kernel security profiles like AppArmor</h3><p>AppArmor is a Linux security module to protect OS and its applications from security threats. Docker provides default profile to allow the program to a limited set of resources like network access, kernel capabilities, and file permissions, etc. It reduces the potential attack surface and provides a great in-depth defense.</p><h3>9. Secure centralized and remote logging</h3><p>Usually, containers log everything on STDOUT, and these logs are lost, once they are terminated, it is important to securely stream the logs to a centralized system for audit and future forensics. We also need to ensure that this logging system is secured and that there is no data leakage from the logs.</p><h3>10. Deploy runtime security monitoring</h3><p>Even if you deploy vulnerability scanning solutions based on repository data and take all necessary precautions, there is still a chance of being victimized. It is important to continuously <a href="https://www.infracloud.io/observability-consulting/">monitor and log the application behavior</a> to prevent and detect malicious activities.</p><blockquote>“There is no silver bullet solution with Cyber Security, a layered defence is the only viable defence.” — ICIT Research</blockquote><p>By implementing the above best practices, you can make it harder for the attacker to find ways to exploit your system. I am pointing out some tools and references that can be used to audit and secure the containers. Security is a vast topic, we haven’t covered Kubernetes specific controls in this article but stay tuned, we can have a follow-up article focussing on the Kubernetes security best practices.</p><h3>Tools</h3><p>To simplify the adoption of security controls, I am suggesting a few open source and commercial offerings which can be used to discover the current state, to generate advisories for your workload.</p><ul><li><a href="https://github.com/docker/docker-bench-security">docker-bench-security</a> — Official tool by Docker itself to audit the container workload according to the <a href="https://www.cisecurity.org/benchmark/docker/">CIS Benchmark</a> for Docker which is an industry-standard benchmark.</li><li><a href="https://github.com/hadolint/hadolint">Hadolint Linter for Dockerfile</a> — Use the linter to do static code analysis of the Dockerfile. The linter helps in implementing the best practices. It can be integrated with popular code editors and integration pipelines.</li><li><a href="https://github.com/quay/clair">Clair</a> — Clair is a popular static vulnerability scanning tool for the application container. It sources metadata from the various vulnerability databases on a regular basis. Alternatives are <a href="https://anchore.com/opensource/">Anchore</a>, <a href="http://synk.io/">Synk</a>, <a href="https://github.com/aquasecurity/trivy">Trivy</a>.</li><li><a href="https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html">OWASP Cheatsheet</a> — OWASP is an open community which is quite popular among security experts. This cheat sheet is a good starting point to start with.</li><li>OpenSCAP for Container — Security Content Automation Protocol (SCAP) is a multi-purpose framework of specifications that supports automated configuration, vulnerability and patch checking, technical control compliance activities, and security measurement. It implements NIST standards.</li><li><a href="https://sysdig.com/opensource/falco/">Sysdig Falco</a> — Falco can be used to implement runtime security. It uses efficient eBPF to intercept calls and traffic for real-time monitoring and forensics. As hackers continue to evolve, new vulnerabilities are discovered and often not picked up by static scanning tools. A solution with machine learning capability, continuous behavioral monitoring, and advanced AI/ML-based engines can’t be ignored from the list of essentials. Learn more about <a href="https://www.infracloud.io/blogs/introduction-kubernetes-security-falco/">getting started Kuberentes security using Falco in this handy blog post.</a></li><li>Commercial offerings from AquaSec, Twistlock, Sysdig, Synk, Qualys for Enterprise-grade security tools, and solutions.</li></ul><h3>References</h3><ol><li><a href="https://www.openshift.com/blog/container-image-security-beyond-vulnerability-scanning">Container Image Security: Beyond Vulnerability Scanning</a></li><li><a href="https://blog.aquasec.com/docker-security-best-practices">Top 20 Docker Security Best Practices: Ultimate Guide</a></li><li><a href="https://techbeacon.com/security/10-top-open-source-tools-docker-security">10+ top open-source tools for Docker security</a></li><li><a href="https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdf">NIST Application Container Security</a></li><li><a href="https://www.cisecurity.org/benchmark/docker/">CIS Benchmark for Docker</a></li><li><a href="https://cloud.google.com/solutions/best-practices-for-building-containers">Best practices for building containers</a></li></ol><p>Hope you enjoyed the post. Do share your thoughts by starting a conversation on <a href="https://www.twitter.com/anjuls">Twitter</a> if you have any interesting security incident or a preventable hack involving Containers you want to share with the community.</p><p>Note — This article is originally authored by me at <a href="https://www.infracloud.io/blogs/top-10-things-for-container-security/">InfraCloud Blogs</a>.</p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=317d8070b9a9" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/top-10-container-security-best-practices-317d8070b9a9">Top 10 Container Security Best Practices</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Shifting from Traditional Infra to Cloud Native Role]]></title>
            <link>https://faun.pub/changing-gears-joining-lummo-d1e60fc382d3?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1e60fc382d3</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[careers]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Thu, 10 Mar 2022 07:52:17 GMT</pubDate>
            <atom:updated>2022-03-11T18:38:08.337Z</atom:updated>
            <content:encoded><![CDATA[<h3>Changing Gears — Shifting from being a DBA to a Cloud-Native Architect</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j01RVn9jtMeEEdY3" /><figcaption>Photo by <a href="https://unsplash.com/@helloimnik?utm_source=medium&amp;utm_medium=referral">Hello I&#39;m Nik</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>For a long time, I wanted to share my story with everyone. So finally, I got some time to jot down to specifically talk about how to build a career in Cloud-native technologies?</p><p>I have spent over a decade working in IT and Technology companies. Worked for large enterprises — Telco, Banks, and Products.</p><p>In 2007, when I was doing my engineering course, I got the offer from Accenture to join them as an Associate. In 2008, I finished college with excitement to join Accenture.</p><p>I worked in a traditional Infrastructure team for more than a decade and then moved to DevOps as a career. The switch was technically not hard, but a few things to keep in mind so that it is easy from the organization&#39;s perspective.</p><ol><li>Find a mentor who is into the new skills and use your existing network to get the job. Often, it is hard to prove that you can take up the new skill in a new company, so sticking with an existing project or company is usually easy.</li><li>Work on improving trust and showing curiosity all the time. It helps your manager to take more risks in your career change.</li><li>Self-learning: Be a continuous learner and try to mix skills in your existing role.</li><li>Self-Branding: If you want to do Y and always brand yourself as X, it won’t help. Change your resume, LinkedIn profile and start promoting yourself in the newer areas. Join meetups, try to write about the things you want to do, and improve your network. The next opportunity mostly comes from your soft skills rather than your technical skills.</li></ol><h3>Start of the Career</h3><p>In 2007, when I was doing my engineering course, I got the offer from Accenture to join them as an Associate. In 2008, I finished college with excitement to join Accenture.</p><p>Initially, for six months, learning was slow due to the global recession and not having projects to work on. I continued to explore databases and found some excellent mentors who taught me very well. It went well, and I got to work in a very large Telco environment, setting up things from scratch, learning from senior DBAs, and awesome managers who believed in me. Fast-forward, this went really well, and I got the opportunity to taste WFH for a couple of years in 2011–12.</p><p>I enjoyed every day doing performance tuning, mentoring developers on how to write performant queries, doing code reviews, automating mundane tasks, evaluating, and improving database performance. This was the most productive time of my life.</p><h3>Standing on the Shoulders of the Giants</h3><p>In the mid of 2012, I moved to Melbourne, sitting closely with the Client Operations team and senior members of my own team. I got guidance from seniors in doing deep analysis and my work started getting attention. A lot of people across the organization engaged me for deep troubleshooting, planning, long term performance improvements. This allowed me to work on various types of applications and tech stacks such as billing systems, CRM, Integration layers, ETLs, design of disaster recovery sites, and large-scale data processing pipelines. During the course, I regularly interacted with the star DBAs and database engineers across the world through internal and external engagements and sessions.</p><p>I kept my interest open to all types of technologies and skills and tried to mix them to improve my productivity of myself, increase my knowledge, and reduce the boredom of remaining in the same skill for a long time. Just to give an example, I used to create automation and self-service in PHP/Shell, used language R to do analysis and create great visuals of performance problems; used Splunk to create capacity reports with future predictions using its own ML models. I also started putting my deep knowledge in Oracle and turned that into tooling and shared it with other fellow teams. This freed up my time to do higher-level functions and take up more challenging tasks.</p><blockquote>Standing on the shoulders of the giant is the quickest way to upskill yourselves.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EMfnq367ukkjW_6u" /><figcaption>Photo by <a href="https://unsplash.com/@chuttersnap?utm_source=medium&amp;utm_medium=referral">CHUTTERSNAP</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Shifting to Cloud and Kubernetes</h3><p>I knew that the DBA skills would soon be automated with the growing Cloud initiatives, and most of the knowledge was automated by the product and through tooling itself. I started looking for a change and at that time in 2018, luckily, I got an opportunity to design a Microservices platform for a logistics and warehouse system. However, this also required me to use my existing DBA skills to design a DB2 cluster. I could not say no to this because I was getting a mix of legacy and next-generation systems.</p><p>I barely heard of Kubernetes at that time. Furthermore, I tried the upstream Kubernetes installation on some VMs in AWS and started exploring the Kubernetes groups and forums for all my problems in the first week. By the end of the week, I started finding groups and people who were doing really well. I would like to thank the Kubernetes community for being such a warm and open place. Any newbie can actually start talking to more experienced folks on their forums and allow you to access all the information needed for you to solve the problems.</p><p>I participated in their community meetings and had access to their groups, where I got a vast amount of first-hand information and knowledge.</p><p>In March 2018, I volunteered for <a href="https://2020.container.camp/web">Container Camp</a> Australia where I met Kubernetes experts from various organizations like Google, Heptio, SUSE, AWS, iflix, ARM, etc, and got a chance to personally talk to <a href="https://dave.cheney.net/"><em>Dave Cheney</em></a>. He is the author of Contour — Ingress controller for Kubernetes. It was a very inspiring few minutes that made a lasting impact on me.</p><p>Since then, I have explored various cloud-native technologies and started following all the conferences and meetups. These platforms allowed me to network with others and learn from them.</p><p>In 2020, I started looking for a job in India, and naturally, I was inclined towards cloud-native tech. I shortlisted the core contributors of cloud-native projects from India such as InfraCloud, Platform 9, Mayadata etc, who had been doing really well in this space. I got to know about <a href="https://www.linkedin.com/in/vishalbiyani/">Vishal</a> from the Container Camp talks, who is the CTO of InfraCloud. I reached out to him on LinkedIn and had a conversation about what they were doing and in the same week, they offered me a Cloud-Native Solution Architect role at InfraCloud. Furthermore, I was really impressed by Vishal and his thinking and ability to innovate in the cloud-native industry.</p><h3>InfraCloud</h3><p>There are a few reasons, why I joined InfraCloud:</p><ul><li>Wanted to work in a fast-growing startup that are innovators and thought leaders.</li><li>Clients were product companies designing something cool using cloud-native tech or Kubernetes-related product itself.</li><li>Remote, always improving, and risk-taking.</li><li>Great culture — you can read more about it on <a href="https://www.infracloud.io/blogs/category/culture/">their blog</a>.</li></ul><p>All of the above were true, and I worked closely with Vishal on various initiatives — be it growing the organization or solving complex technical problems.</p><p>I have provided consulting to various small to large organizations in the first year and also participated in various internal initiatives to grow the company. I worked closely with the HR team, talent acquisition, marketing, and potential leads.</p><p>At InfraCloud, I spent 1.5 years grooming myself into cloud-native tech, technical leadership, <a href="https://anjul.dev/writings">technical writing</a>, and <a href="https://anjul.dev/mentoring">mentoring</a>. I highly recommend it for someone who is trying to grow themselves in these skills.</p><h3>Lummo (formerly BukuKas &amp; Tokko)</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FzAG_aBqU7ZA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DzAG_aBqU7ZA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FzAG_aBqU7ZA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef1e00e5087b19d72de418b77d55f880/href">https://medium.com/media/ef1e00e5087b19d72de418b77d55f880/href</a></iframe><p>At the end of 2021, I was looking for Internet-scale businesses to see the concerns that you don’t typically see in smaller setups. On this journey, I got to know about <a href="https://lummo.com/"><strong>Lummo</strong></a>, a fast-growing Indonesian market with global aspirations, with a vision with which I can relate myself. I joined them as a Principal Software Engineer in the Platform engineering team. We are building the platform to help SEA entrepreneurs digitize, sell and grow their products and services. We are growing and expanding our teams and working on the latest tech.</p><h3>We are hiring!</h3><p>If you are interested in learning more about us, feel free to <a href="https://calendly.com/anjulsahu/mentoring">book a time</a> with me.</p><figure><img alt="Cloud Native Weekly" src="https://cdn-images-1.medium.com/max/1024/1*8gy7WyVJrxgXb6BxlncXIA.jpeg" /><figcaption>https://anjulsahu.substack.com</figcaption></figure><p>I also publish a newsletter on cloud-native technologies. <a href="http://anjulsahu.substack.com/">Subscribe</a> to receive the latest news and articles in your inbox.</p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1e60fc382d3" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/changing-gears-joining-lummo-d1e60fc382d3">Shifting from Traditional Infra to Cloud Native Role</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps Roadmap 2022]]></title>
            <link>https://faun.pub/devops-roadmap-2022-340934d360f9?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/340934d360f9</guid>
            <category><![CDATA[mentoring]]></category>
            <category><![CDATA[roadmaps]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Thu, 17 Feb 2022 04:50:01 GMT</pubDate>
            <atom:updated>2023-01-24T18:58:58.885Z</atom:updated>
            <content:encoded><![CDATA[<p>In the last few weeks, I met some folks in my <a href="https://anjul.dev/mentoring">mentoring sessions</a>, who are new to DevOps or in the mid of their career, who were interested in knowing what to learn in 2022. DevOps skills are high in demand and there is constant learning required to keep yourself in sync with market demand.</p><p>This post is to share the notes that can help you. Let’s see some guidance based on my experience and understanding.</p><h3>Roadmap</h3><h3>Be fundamentally strong in the networking technologies</h3><p>Understand the concepts such as HTTP/2, QUIC or HTTP3, Layer 4 and Layer 7 protocols, mTLS, Proxies, DNS, BGP, how load balancing works, IP Tables, the working of Internet, IP addresses and schemes, and lastly the Network design. I found <a href="https://jvns.ca/">Julia Evans’s</a> blog very useful and my go-to place when I need to understand stuff in a simple way. She has covered a wide variety of topics in her blog posts and zines.</p><h3>Master the operating system fundamentals, particularly Linux</h3><p>As most of the systems (VMs, Containers, etc) run Linux, it is important to know from top to bottom. Learn scheduling, systemd interface, init system, cgroups and namespaces, performance tuning, and mastering the command line utilities — awk, sed, jq, yq, curl, ssh, openssl etc., Learn performance troubleshooting from <a href="https://www.brendangregg.com/">Brendan’s blog</a>.</p><h3>CI/CD</h3><p>If you are still into Jenkins, it is fine. But, the world has moved to cloud-native pipelines. Conceptually not much has changed in this space, but you can look into Github Actions, Tekton etc. How to do releases better? Understand various deployment strategies such as blue-green and canary.</p><h3>Containerisation and Virtualisation</h3><p>Apart from the popular Docker runtime, try containerd, podman etc and knowing How to containerise applications, how to implement <a href="https://anjul.dev/blog/top-10-things-for-container-security/">container security</a>, and how to run and orchestrate VMs in Kubernetes, see KubeVirt project.</p><h3>Container Orchestration</h3><p>Kubernetes is now a de facto standard for running containers. There is a lot of content on the Internet to learn Kubernetes. Focus on configuration best practices, application design, security, and scheduling. Setting up a cluster is getting trivial now but the day 2 operational stuff such as setting up, monitoring, logging, CI/CD, how to scale the cluster, cost optimization, and security are some questions people might be expecting from you.</p><p>Get a big picture of what is underneath the Kubernetes iceberg in this <a href="https://asankov.dev/blog/2022/05/15/demystifying-the-kubernetes-iceberg-part-1/">article</a> by Asankov. It is a series of articles that explores Kubernetes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*lWfto5GnwEerYA2f.png" /><figcaption>Source: asankov.dev</figcaption></figure><h3>Observability at Scale</h3><p>Most of the engineers are aware of the Prometheus Grafana stack or similar. Trends suggests that many organisations are consolidating their Kubernetes clusters and observability, both from the performance and cost perspective, this helps. Learn the advanced configuration and architectures of Prometheus, and how to scale them. Look into technologies like Thanos, Cortex, VictoriaMetrics, Datadog, and Loki. Continuous profiling tools such as Parca, periscope, hypertrace and distributed tracing with open telemetry. Service meshes such as Istio are popular ingredients in cloud-native recipes.</p><h3>Platform team as a Product team</h3><p>The function of the Platform team is becoming more like a centralized product team that is focussing on its internal platform customers such as Developers and testers. The goal is to improve the ways of working and bring some order to the teams. Try to improvise on the problems the Developer and QA team faces. You are the enabler for other teams, instead of taking all the work in a central team, coach the dev team to take up typical DevOps responsibilities. That way you can scale and don’t burn yourself too much.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c0-Zb_IhN01m_meq.png" /></figure><h3>Security</h3><p>In many small organizations, security was a second-class citizen. Product features were given more priority. But, due to growing sophisticated attacks and various strict compliance requirements, companies are adapting to shift-left security strategies. End-to-end encryption, strong RBAC, IAM policies, governance and auditing, and implementation of benchmarks such as NIST, CIS, ISO27001 are common. Container security, Policy as code, Cloud Governance, and Supply chain security are hot topics.</p><h3>Programming</h3><p>DevOps or SRE role is now taking the cross-cutting concerns of the Developers and creating tooling that can help in improving their productivity while enforcing the standards. A good quality software engineering practice and skills are required to craft high-quality platform components.</p><p>I can’t give enough stress to this. Good organizations are looking for good programming experience in Platform engineers. It is important in site reliability engineering as well, where you need to be fluent in programming, able to read, understand and debug the code written by others and if necessary, fix it.</p><p>Python and Golang are the most popular ones. My suggestion is Golang due to features like strong concurrency, strict type checking, adoption in various organs, toolchain and as many major projects are built using Golang, it makes sense to learn that over Python.</p><p>A few simple things you can try:</p><ul><li>Write a CLI in your programming language.</li><li>Learn to write a REST API and interact with databases</li><li>Parallelism and Concurrency</li></ul><h3>Infrastructure as Code</h3><p>Terraform is a standard in the projects. Once you understand the concept, it is easy to adapt to any other tooling as most of them are based on DSL.</p><h3>Cloud</h3><p>Most of the cloud works in the same way. So if you know one cloud well, you can easily work with other cloud providers. Focus on how you can design applications using cloud-native components in a highly available, resilient, secured, and cost-effective way.</p><h3>Technical Writing</h3><p>You might be wondering why I am talking about technical writing when discussing DevOps. A lot of folks don’t give enough attention to this, but it is super important in how you communicate and work with other teams. The future of work is remote and emails, slack/teams, and chats are the primary channels to talk and convey ideas to others.</p><p>On a regular basis, you might be creating documents such as runbooks, postmortems, RFCs, architectural decision records, and software design docs to name a few. A clear, easy-to-understand document does wonders. It can help you save your and the reader’s time and improve overall productivity. Suggest you read <a href="https://blog.pragmaticengineer.com/becoming-a-better-writer-in-tech/">this article</a>.</p><h3>Site Reliability Engineering</h3><p>The boundary between DevOps and SRE is getting thin. In some organisations, the same person might be performing both roles. Understand the concepts behind SLI, SLO, and Error budgets and SRE practices. Each organisation does it differently, so I don’t recommend copy-paste someone else’s culture into your team. Refer to the <a href="https://sre.google/">Google SRE culture</a>.</p><h3>Conclusion</h3><p>Personally, I am excited about following in this year. This is not a definitive list as it keeps changing with time.</p><ul><li>Service Mesh — Istio, Cilium Sidecarless mesh, Tetrate and Solo’s Gloo mesh offering.</li><li>How to improve Developer Productivity? It is a mix of culture, automation and tools.</li><li>SRE Platforms — <a href="https://honeycomb.io/">honeycomb</a>, <a href="https://last9.io/">Last9</a>.</li><li>DevPortals — again linked with the motive of improving productivity and bridging knowledge gap.</li><li>Observability — technologies such as open telemetry, hypertrace, <a href="https://thanos.io/">Thanos</a>, VictoriaMetrics, <a href="https://vector.dev/">Vector</a>.</li><li>Security — supply chain security, code signing, tightening cloud security.</li><li>Golang — improving the current skills.</li><li>Serverless computing and Event-driven architectures</li><li>Web3 — understanding the landscape related to DevOps and Infrastructure</li></ul><p>Be curious and keep learning. Continuous bite-size learning is easy, which you can do along with your full-time job. If you still have any questions, feel free to <a href="https://anjul.dev/">book some time </a>with me. I am more than happy to help.</p><p>If you are looking for professional consulting in cloud-native and Kubernetes, reach out to us at <a href="https://cloudraft.io/contact-us">https://cloudraft.io/contact-us</a>.</p><p><a href="https://anjulsahu.substack.com/">Cloud Native Weekly</a></p><figure><a href="https://faun.to/bP1m5"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCiLLad3dvZLwBa-B5cAVQ.png" /></a></figure><p>Edits:</p><ul><li>7 July 2022 — added Demystifying Kubernetes series in the Container orchestration section.</li></ul><p>Join FAUN: <a href="https://faun.to/i9Pt9"><strong>Website</strong></a><strong> </strong>💻<strong>|</strong><a href="https://faun.dev/podcast"><strong>Podcast</strong></a><strong> </strong>🎙️<strong>|</strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong>|</strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong>|</strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷|<a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook Group</strong></a><strong> </strong>🗣️<strong>|</strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin Group</strong></a><strong> </strong>💬<strong>|</strong> <a href="https://faun.dev/chat"><strong>Slack</strong></a> 📱<strong>|</strong><a href="https://thechief.io/"><strong>Cloud Native</strong> <strong>News</strong></a><strong> </strong>📰<strong>|</strong><a href="https://linktr.ee/faun.dev/"><strong>More</strong></a><strong>.</strong></p><p><strong>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=340934d360f9" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/devops-roadmap-2022-340934d360f9">DevOps Roadmap 2022</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning Orchestration on Kubernetes using Kubeflow]]></title>
            <link>https://anjuls.medium.com/machine-learning-orchestration-on-kubernetes-using-kubeflow-2069767e20ca?source=rss-808537596ad5------2</link>
            <guid isPermaLink="false">https://medium.com/p/2069767e20ca</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[kubeflow]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Anjul Sahu]]></dc:creator>
            <pubDate>Thu, 01 Apr 2021 10:17:22 GMT</pubDate>
            <atom:updated>2021-04-01T10:17:22.005Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HR2ALjvoqzEzDjt_kraedw.png" /></figure><h3>MLOps: From Proof Of Concepts to Industrialization</h3><p>In recent years, AI and Machine Learning have seen tremendous growth across industries in various innovative use cases. It is the most important strategic trend for business leaders. When we dive into a technology, the first step is usually experimentation on a small scale and for very basic use cases, then the next step is to scale up operations. Sophisticated ML models help companies efficiently discover patterns, uncover anomalies, make predictions and decisions, and generate insights, and are increasingly becoming a key differentiator in the marketplace. Companies recognise the need to move from proof of concepts to engineered solutions, and to move ML models from development to production. There is a lack of consistency in tools and the development and deployment process is inefficient. As these technologies mature, we need operational discipline and sophisticated workflows to take advantage and operate at scale. This is popularly known as MLOps or ML CI/ CD or ML DevOps. In this article, we explore how this can be achieved with the Kubeflow project, which makes deploying machine learning workflows on Kubernetes simple, portable, and scalable.</p><h3>MLOps in Cloud Native World</h3><p>There are Enterprise ML platforms like Amazon SageMaker, Azure ML, Google Cloud AI, and IBM Watson Studio in public cloud environments. In case of on-prem and hybrid open source platform, the most notable project is Kubeflow.</p><h3>What is Kubeflow?</h3><p>Kubeflow is a curated collection of machine learning frameworks and tools. It is a platform for data scientists and ML engineers who want to experiment with their model and design an efficient workflow to develop, test and deploy at scale. It is a portable, scalable, and open-source platform built on top of Kubernetes by abstracting the underlying Kubernetes concepts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dRB7yE1Wz0yXxa7L.png" /></figure><h3>Kubeflow Architecture</h3><p>Kubeflow utilizes various cloud native technologies like Istio, Knative, Argo, Tekton, and leverage Kubernetes primitives such as deployments, services, and custom resources. Istio and Knative help provide capabilities like blue/green deployments, traffic splitting, canary releases, and auto-scaling. Kubeflow abstracts the Kubernetes components by providing UI, CLI, and easy workflows that non-kubernetes users can use.</p><p>For the ML capabilities, Kubeflow integrates the best framework and tools such as TensorFlow, MXNet, Jupyter Notebooks, PyTorch, and Seldon Core. This integration provides data preparation, training, and serving capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kVILgUxK_EyOTmMpBDUffw.png" /><figcaption>Kubeflow Architecture</figcaption></figure><h3>Let’s look at Kubeflow Components</h3><ul><li><strong>Central Dashboard</strong>: User interface for managing all the Kubeflow pipeline and interacting with various components.</li><li><strong>Jupyter Notebooks</strong>: It allows to collaborate with other team members and develop the model.</li><li><strong>Metadata</strong> — It helps in organizing workflows by tracking and managing the metadata in the artifacts. In this context, metadata means information about executions (runs), models, datasets, and other artifacts. Artifacts are the files and objects that form the inputs and outputs of the components in your ML workflow.</li><li><strong>Fairing</strong>: It allows running training job remotely by embedding it in Notebook or local python code and deploy the prediction endpoints.</li><li><strong>Feature Store (Feast)</strong>: It helps in feature sharing and reuse, serving features at scale, providing consistency between training and serving, point-in-time correctness, maintaining data quality and validation.</li><li><strong>ML Frameworks</strong>: This is a collection of frameworks including, Chainer (deprecated), MPI, MXNet, PyTorch, TensorFlow, providing</li><li><strong>Katib</strong>: It is used to implement Automated machine learning using Hyperparameters (variables to control the model training process), Neural Architecture Search (NAS) to improve predictive accurancy and performance of the model, and a web UI to interact with Katib.</li><li><strong>Pipelines</strong>: Provides end-to-end orchestration and easy to reuse solution to ease the experimentations.</li><li><strong>Tools for Serving</strong>: There are two model serving systems that allow multi-framework model serving: KFServing, and Seldon Core. You can read more <a href="https://www.kubeflow.org/docs/components/serving/overview/">about tools for serving here</a>.</li></ul><h3>What are some of the Kubeflow Use Cases?</h3><ul><li><strong>Hybrid multi-cloud ML Platform at scale</strong>: As Kubeflow is based on Kubernetes, it utilized all the features and power that Kubernetes provides. This allows you to design ML platforms that are portable and utilize the same APIs etc. to run on on-prem and public clouds.</li><li><strong>Experimentation</strong>: Easy UI and abstraction help in rapid experimentation and collaboration. This speeds development by providing guided user journeys.</li><li><strong>DevOps for ML platform</strong>: Kubeflow pipelines can help creating reproducible workflows which deliver consistency, save iteration time, and helps in debugging, auditability, and compliance requirements.</li><li><strong>Tuning the model hyperparameters during training</strong>: During model development, hyperparameters tuning is often hard to tune and time-consuming. It is also critical for model performance and accuracy. Katib can reduce the testing time and improve the delivery speed by automating hyperparameters tuning.</li></ul><h3>Kubeflow Demo</h3><p>Let’s try to learn Kubeflow with an example. In this demo, we will try Kubeflow on a local Kind cluster. You should have at least 16GB of RAM, 8 CPUs modern machine to try it on your local machine, otherwise use a VM in the cloud. We will use Zalando’s fashion MNIST dataset and <a href="https://github.com/anjuls/fashion-mnist-kfp-lab/blob/master/KF_Fashion_MNIST.ipynb">this notebook by <em>manceps</em></a> for demo.</p><p>Due to some <a href="https://github.com/kubeflow/kfctl/issues/406">issue</a>, I had to enable few feature gates and extra API server arguments to make it work. Please use the following Kind configuration to create the cluster.</p><pre># kind cluster configuration - kind.yaml</pre><pre>kind: Cluster<br>apiVersion: kind.x-k8s.io/v1alpha4<br>featureGates:<br>  &quot;TokenRequest&quot;: true<br>  &quot;TokenRequestProjection&quot;: true<br>kubeadmConfigPatches:<br>  - |<br>    kind: ClusterConfiguration<br>    metadata:<br>      name: config<br>    apiServer:<br>      extraArgs:<br>        &quot;service-account-signing-key-file&quot;: &quot;/etc/kubernetes/pki/sa.key&quot;<br>        &quot;service-account-issuer&quot;: &quot;kubernetes.default.svc&quot;</pre><p>Create the Kind cluster and install Kubeflow.</p><pre># Create Kind cluster<br>kind create cluster --config kind.yaml<br></pre><pre># Deploy Kubeflow on Kind. </pre><pre>mkdir -p /root/kubeflow/v1.0<br>cd /root/kubeflow/v1.0<br>wget https://github.com/kubeflow/kfctl/releases/download/v1.0/kfctl_v1.0-0-g94c35cf_linux.tar.gz</pre><pre>tar -xvf kfctl_v1.0-0-g94c35cf_linux.tar.gz			<br>export PATH=$PATH:/root/kubeflow/v1.0<br>export KF_NAME=my-kubeflow<br>export BASE_DIR=/root/kubeflow/v1.0<br>export KF_DIR=${BASE_DIR}/${KF_NAME}<br>export CONFIG_URI=&quot;https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_k8s_istio.v1.2.0.yaml&quot; </pre><pre>mkdir -p ${KF_DIR}<br>cd ${KF_DIR}<br>kfctl apply -f ${CONFIG_URI}</pre><p>It may take 15–20 minutes to bring up all the services.</p><pre>❯ kubectl get pods -n kubeflow<br>NAME                                                     READY   STATUS    RESTARTS   AGE<br>admission-webhook-bootstrap-stateful-set-0               1/1     Running   0          19m<br>admission-webhook-deployment-5cd7dc96f5-4hsqr            1/1     Running   0          18m<br>application-controller-stateful-set-0                    1/1     Running   0          21m<br>argo-ui-65df8c7c84-dcm6m                                 1/1     Running   0          18m<br>cache-deployer-deployment-5f4979f45-6fvg2                2/2     Running   1          3m21s<br>cache-server-7859fd67f5-982mg                            2/2     Running   0          102s<br>centraldashboard-67767584dc-f5zhh                        1/1     Running   0          18m<br>jupyter-web-app-deployment-8486d5ffff-4cb8n              1/1     Running   0          18m<br>katib-controller-7fcc95676b-brk2q                        1/1     Running   0          18m<br>katib-db-manager-85db457c64-bb7dp                        1/1     Running   3          18m<br>katib-mysql-6c7f7fb869-c4qqx                             1/1     Running   0          18m<br>katib-ui-65dc4cf6f5-qrjpm                                1/1     Running   0          18m<br>kfserving-controller-manager-0                           2/2     Running   0          18m<br>kubeflow-pipelines-profile-controller-797fb44db9-hdnxc   1/1     Running   0          18m<br>metacontroller-0                                         1/1     Running   0          19m<br>metadata-db-6dd978c5b-wtglv                              1/1     Running   0          18m<br>metadata-envoy-deployment-67bd5954c-z8qrv                1/1     Running   0          18m<br>metadata-grpc-deployment-577c67c96f-ts9v6                1/1     Running   6          18m<br>metadata-writer-756dbdd478-7cbgj                         2/2     Running   0          18m<br>minio-54d995c97b-85xl6                                   1/1     Running   0          18m<br>ml-pipeline-7c56db5db9-9mswf                             2/2     Running   0          18s<br>ml-pipeline-persistenceagent-d984c9585-82qvs             2/2     Running   0          18m<br>ml-pipeline-scheduledworkflow-5ccf4c9fcc-mjrwz           2/2     Running   0          18m<br>ml-pipeline-ui-7ddcd74489-jw8gj                          2/2     Running   0          18m<br>ml-pipeline-viewer-crd-56c68f6c85-tszc4                  2/2     Running   1          18m<br>ml-pipeline-visualizationserver-5b9bd8f6bf-dj2r6         2/2     Running   0          18m<br>mpi-operator-d5bfb8489-9jzsf                             1/1     Running   0          4m27s<br>mxnet-operator-7576d697d6-7wj52                          1/1     Running   0          18m<br>mysql-74f8f99bc8-fddww                                   2/2     Running   0          18m<br>notebook-controller-deployment-5bb6bdbd6d-vx8tv          1/1     Running   0          18m<br>profiles-deployment-56bc5d7dcb-8x7vr                     2/2     Running   0          18m<br>pytorch-operator-847c8d55d8-zgh2x                        1/1     Running   0          18m<br>seldon-controller-manager-6bf8b45656-6k8r7               1/1     Running   0          18m<br>spark-operatorsparkoperator-fdfbfd99-5drsc               1/1     Running   0          19m<br>spartakus-volunteer-558f8bfd47-h2w62                     1/1     Running   0          18m<br>tf-job-operator-58477797f8-86z42                         1/1     Running   0          18m<br>workflow-controller-64fd7cffc5-77g6z                     1/1     Running   0          18m</pre><p>Now, you can access the Kubeflow dashboard by port-forwarding on http2/$INGRESS_PORT where can be fetched using the below.</p><pre>export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath=&#39;{.spec.ports[?(@.name==&quot;http2&quot;)].nodePort}&#39;)</pre><h3>Let’s Try an Experiment</h3><p>We will be using Zalando’s Fashion-MNIST dataset to show basic classification using Tensorflow in this experiment.</p><p><strong>About the Dataset</strong> Fashion-MNIST is a dataset of Zalando’s article images — consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image associated with a label from 10 classes. We intend Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the exact image size and structure of training and testing splits. <em>source</em>: <a href="https://github.com/zalandoresearch/fashion-mnist">https://github.com/zalandoresearch/fashion-mnist</a></p><p>The whole experiment is sourced from <em>manceps</em> notebook. Create a Jupyter notebook with the name kf-demo using <a href="https://github.com/anjuls/fashion-mnist-kfp-lab/blob/master/KF_Fashion_MNIST.ipynb">this notebook</a>.</p><p>You can run the notebook from the dashboard and create the pipeline. Please note, in Kubeflow v1.2, there is an issue causing RBAC: permission denied error while connecting to the pipeline. This will be fixed in v1.3 and you can read more about the issue <a href="https://github.com/kubeflow/pipelines/issues/4440">here</a>. As a workaround, you need to create Istio ServiceRoleBinding and EnvoyFilter to add an identity in the header. <a href="https://gist.github.com/anjuls/43c7642ddb8be46c9d6095503a296862">Refer to this gist</a> for the <strong>patch</strong>.</p><p>The Kubeflow will orchestrate various components to create the pipeline and run the ML experiment. You can access the results through the dashboard. Behind the scene, Kubernetes pods, argo workflows, etc., are created which you don’t need to worry about.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hWpZ-byo-Kz2cWy4.png" /></figure><p><em>Pods running the kf-demo notebook and pipeline</em></p><p>I also noticed that when running the Pipeline in Kind, it complained about the following:</p><pre>MountVolume.SetUp failed for volume &quot;docker-sock&quot; : hostPath type check<br>       failed: /var/run/docker.sock is not a socket file</pre><p>To resolve this, I had to change the Argo Workflow ConfigMap to use pns instead of docker container runtime executor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y81gzNhz5R05phQA.png" /></figure><p>After the change, re-run the experiment from the dashboard, which will then pass through.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TSmz5tC12jtIryMx.png" /></figure><p><em>Experiment Flow</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kDz3IPKu7d_2c8Jl.png" /></figure><p><em>Prediction Result</em></p><h3>Conclusion</h3><p>If you are looking for bringing agility, improved management with enterprise-grade features such as RBAC, multi-tenancy and isolation, security, auditability, collaboration for the machine learning operations in your organization, Kubeflow is an excellent option. It is stable, mature and curated with best-in-class tools and framework which can be deployed in any Kubernetes distribution. See <a href="https://github.com/kubeflow/kubeflow/blob/master/ROADMAP.md">Kubeflow roadmap here</a> to look into what’s coming in the next version.</p><p>Hope this was helpful to you. Do try Kubeflow and share your experience by connecting with me on <a href="https://twitter.com/anjuls">Twitter</a>.</p><h3>References</h3><ol><li><a href="https://www.gartner.com/smarterwithgartner/gartner-top-strategic-technology-trends-for-2021/">https://www.gartner.com/smarterwithgartner/gartner-top-strategic-technology-trends-for-2021/</a></li><li><a href="https://www2.deloitte.com/content/dam/insights/articles/6730_TT-Landing-page/DI_2021-Tech-Trends.pdf">https://www2.deloitte.com/content/dam/insights/articles/6730_TT-Landing-page/DI_2021-Tech-Trends.pdf</a></li><li><a href="https://www.crn.com/news/cloud/5-emerging-ai-and-machine-learning-trends-to-watch-in-2021?itc=refresh">https://www.crn.com/news/cloud/5-emerging-ai-and-machine-learning-trends-to-watch-in-2021?itc=refresh</a></li><li><a href="https://www.cncf.io/blog/2019/07/30/deploy-your-machine-learning-models-with-kubernetes/">https://www.cncf.io/blog/2019/07/30/deploy-your-machine-learning-models-with-kubernetes/</a></li><li><a href="https://events19.linuxfoundation.org/wp-content/uploads/2018/02/OpenFinTech-MLonKube10112018-atin-and-sahdev.pdf">https://events19.linuxfoundation.org/wp-content/uploads/2018/02/OpenFinTech-MLonKube10112018-atin-and-sahdev.pdf</a></li><li><a href="https://thenewstack.io/how-kubernetes-could-orchestrate-machine-learning-pipelines/">https://thenewstack.io/how-kubernetes-could-orchestrate-machine-learning-pipelines/</a></li><li><a href="https://cloud.google.com/community/tutorials/kubernetes-ml-ops">https://cloud.google.com/community/tutorials/kubernetes-ml-ops</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2069767e20ca" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>