<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Harshad Deshmukh on Medium]]></title>
        <description><![CDATA[Stories by Harshad Deshmukh on Medium]]></description>
        <link>https://medium.com/@deshmukhharshad2010?source=rss-23a1e5036160------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Harshad Deshmukh on Medium</title>
            <link>https://medium.com/@deshmukhharshad2010?source=rss-23a1e5036160------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:23:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@deshmukhharshad2010/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ From Docker & Kubernetes to Kubeflow: Scaling ML Workflows Made Easy! ]]></title>
            <link>https://medium.com/@deshmukhharshad2010/from-docker-kubernetes-to-kubeflow-scaling-ml-workflows-made-easy-09678ed46567?source=rss-23a1e5036160------2</link>
            <guid isPermaLink="false">https://medium.com/p/09678ed46567</guid>
            <dc:creator><![CDATA[Harshad Deshmukh]]></dc:creator>
            <pubDate>Tue, 26 Nov 2024 07:42:07 GMT</pubDate>
            <atom:updated>2024-11-26T07:43:46.533Z</atom:updated>
            <content:encoded><![CDATA[<p>In my previous posts on LinkedIn, I explored how <strong>Docker</strong> revolutionized containerization and how <strong>Kubernetes</strong> became the go-to solution for managing containers at scale. But when it comes to <strong>Machine Learning (ML)</strong> workflows, the story doesn’t stop there.</p><p>ML introduces unique challenges that require specialized tools, and this is where <strong>Kubeflow</strong> enters the scene. Let’s explore how it connects to Docker and Kubernetes and why it’s a game-changer for ML practitioners. 🧑‍💻</p><p><strong>🔧 Why Docker &amp; Kubernetes Alone Aren’t Enough</strong></p><p>Docker and Kubernetes are excellent for general application development, but ML workflows have additional requirements:</p><ul><li>Tracking experiments and results 📝</li><li>Building and running complex data pipelines 🔄</li><li>Managing large-scale training jobs ⚙️</li><li>Deploying and monitoring models in production 🔍</li></ul><p>While you can try to solve these with raw Docker and Kubernetes, the effort can become overwhelming. What you need is a platform that integrates ML-specific tools with containerization and orchestration. Enter <strong>Kubeflow</strong>.</p><p><strong>🔄 Kubeflow: The ML Platform on Kubernetes</strong></p><p>Kubeflow is an <strong>open-source</strong> platform designed to simplify and scale ML workflows using Kubernetes. Here’s how it builds on Docker and Kubernetes:</p><ul><li><strong>Containerized Workflows</strong>: Each step of your ML pipeline — data preprocessing, model training, or inference — can be containerized with Docker for consistency and portability.</li><li><strong>Kubernetes Orchestration</strong>: Kubeflow uses Kubernetes to schedule, scale, and manage these containerized workflows across clusters.</li><li><strong>Composability &amp; Scalability</strong>: Pipelines are modular, reusable, and can scale from a local laptop to a multi-node cloud cluster.</li></ul><p>With Kubeflow, you can move seamlessly from experimentation to production while keeping the infrastructure manageable.</p><p><strong>🎛️ Open-Source Kubeflow vs. Managed Kubeflow (Vertex AI)</strong></p><p>While open-source Kubeflow is powerful, it isn’t always easy to set up and maintain. Configuring Kubernetes for ML pipelines requires advanced expertise, and scaling reliably in production can be challenging.</p><p>To address these challenges, cloud platforms introduced <strong>managed versions</strong> of Kubeflow, with Google Cloud’s <strong>Vertex AI</strong> being a prime example.</p><p><strong>💡 Why Managed Kubeflow (Vertex AI) Came Into the Picture</strong></p><p>Managed Kubeflow solutions like Vertex AI offer:</p><ul><li><strong>Ease of Setup</strong>: Pre-configured environments that save time and effort.</li><li><strong>Reliability</strong>: Automatic scaling and failover ensure high availability.</li><li><strong>Cloud Integration</strong>: Vertex AI seamlessly connects with other Google Cloud tools like BigQuery, DataFlow, and AutoML.</li><li><strong>Better Usability</strong>: Enhanced UIs and built-in monitoring simplify the experience for teams.</li></ul><p>Essentially, managed Kubeflow lets you focus on building and deploying models without worrying about managing the underlying infrastructure.</p><p><strong>🌟 Connecting the Dots</strong></p><p>Think of it like this:</p><ul><li><strong>Docker</strong> lets you containerize ML tasks.</li><li><strong>Kubernetes</strong> scales and orchestrates these containers.</li><li><strong>Kubeflow</strong> bridges the gap by tailoring this infrastructure for ML workflows.</li><li><strong>Vertex AI</strong> takes it a step further by abstracting away the complexities of infrastructure management.</li></ul><p>Whether you’re an ML engineer or an MLOps aspirant, understanding Kubeflow’s role in modern ML workflows is key to staying ahead. 🚀</p><p>What challenges have you faced in scaling ML workflows? Have you tried Kubeflow or Vertex AI? Let’s discuss! 💬</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=09678ed46567" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>