<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pranil Dasika on Medium]]></title>
        <description><![CDATA[Stories by Pranil Dasika on Medium]]></description>
        <link>https://medium.com/@pranildasika?source=rss-3b4516808b00------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LYOMlX4KZ2rm9NcpvNoC3A.jpeg</url>
            <title>Stories by Pranil Dasika on Medium</title>
            <link>https://medium.com/@pranildasika?source=rss-3b4516808b00------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 09:03:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pranildasika/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Agentic Applications: Engineering at the Speed of Possibility]]></title>
            <link>https://pranildasika.medium.com/agentic-applications-engineering-at-the-speed-of-possibility-863bf33b94c0?source=rss-3b4516808b00------2</link>
            <guid isPermaLink="false">https://medium.com/p/863bf33b94c0</guid>
            <category><![CDATA[program-management]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[sdlc-methodologies]]></category>
            <dc:creator><![CDATA[Pranil Dasika]]></dc:creator>
            <pubDate>Thu, 18 Sep 2025 13:56:52 GMT</pubDate>
            <atom:updated>2025-09-18T13:56:52.184Z</atom:updated>
            <content:encoded><![CDATA[<p>I’ve been having fascinating conversations with companies that are pushing the frontier of innovation with applied AI over my time building Augur.</p><p>These teams are building applications that don’t just respond to user input — they anticipate needs, make autonomous decisions, and take proactive action. Customer service systems that resolve issues before tickets are filed. Development environments that refactor code as requirements evolve. Financial platforms that automatically adjust strategies based on analyst feedback.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vIOXMjiBqW8-wTciYag2HA.png" /></figure><h3>The Autonomy Paradox</h3><p>Every breakthrough in tech lowers the cost of something but creates complexity somewhere else. Agentic AI has made intelligent decision-making incredibly cheap — but it’s shifted the complexity to coordination and reliability.</p><p>The teams I work with are learning and applying this firsthand. Their autonomous systems need to integrate through protocols like MCP, coordinate with other services, and maintain state across complex workflows. They’re leading this world of rapid autonomous iteration with <em>more</em> engineering discipline, not less.</p><h3>Impact Engineering: Where Business Value Meets Technical Excellence</h3><p>What’s particularly fascinating is how these autonomous systems demand a new approach to engineering — one I call “impact engineering.” Traditional software development optimizes for functionality and performance. Agentic applications require optimization for business impact.</p><p>This means building tighter interlocks between engineering and business teams. When your system is making autonomous decisions that directly affect customer outcomes, you need real-time feedback loops that connect business metrics to technical performance. You need rapid adaptation mechanisms that can adjust system behavior based on user and customer inputs.</p><p>The most successful teams have learned to engineer for value delivery, not just feature delivery. They’re building systems that can measure their own business impact and autonomously optimize for better outcomes.</p><h3>Where Enterprise Rigor Meets AI Innovation</h3><p>This impact focus is driving teams to adopt enterprise-grade engineering patterns while maintaining startup agility:</p><p>Observability becomes business-critical when your system is making autonomous decisions that affect revenue. You need to understand not just system performance, but business performance.</p><p>State management complexity explodes when multiple agents are operating independently but need to coordinate around shared business objectives. The distributed systems patterns we’ve refined over decades suddenly become essential for business continuity.</p><p>Feedback loops need orchestration that can capture, analyze, and act on both technical metrics and business outcomes in real-time while maintaining audit trails and governance.</p><h3>The New SDLC Reality</h3><p>Traditional software development assumes deterministic outcomes and separate business/technical cycles. Agentic systems are probabilistic by nature and require continuous business-technical alignment. This creates fascinating engineering challenges:</p><ul><li>How do you test systems that learn and adapt to changing business conditions?</li><li>How do you maintain consistency when autonomous agents are making decisions that affect business outcomes across distributed infrastructure?</li><li>How do you handle deployment and rollback when your system’s behavior evolves based on business feedback?</li></ul><p>The teams solving this are creating entirely new approaches to software delivery — combining the experimental nature of AI with the reliability patterns of enterprise engineering and the responsiveness required for business impact.</p><h3>What’s Next</h3><p>We’re at an inflection point. The teams that figure out how to build agentic systems with enterprise-grade reliability and business-grade responsiveness won’t just have better AI — they’ll have redefined what software can accomplish.</p><p>The infrastructure and coordination challenges are real, but so is the potential. We’re building systems that can truly augment business capabilities autonomously while continuously optimizing for impact.</p><p>What are you seeing in the agentic development space? I’m curious about the infrastructure patterns emerging from your work.</p><p><em>Building agentic systems? Let’s discuss the engineering challenges: </em><a href="https://calendly.com/pdasika-augur/30min"><em>https://calendly.com/pdasika-augur/30min</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=863bf33b94c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Day a Customer’s Production System Collapsed (And Why I Built Mantis)]]></title>
            <link>https://pranildasika.medium.com/the-day-a-customers-production-system-collapsed-and-why-i-built-mantis-4f8794b424b7?source=rss-3b4516808b00------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f8794b424b7</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Pranil Dasika]]></dc:creator>
            <pubDate>Tue, 29 Oct 2024 15:31:52 GMT</pubDate>
            <atom:updated>2024-10-29T15:31:52.605Z</atom:updated>
            <content:encoded><![CDATA[<p>Three days. 72 sleepless hours. A corrupted MongoDB instance with a simple configuration error. Millions of dollars at stake. That’s what broke me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0vthhLXNtQnEwooE.png" /></figure><p>My team was responsible for deployment of over 30 microservices across multiple clouds, directly in our enterprise on-prem customers’ environments. We thought we had it under control. We were SO wrong!.</p><p>The reality hit hard: Hour-long deployments stretched into months. Security reviews became endless loops. Integration issues multiplied exponentially. Key stakeholders vanished when we needed them most.</p><p>But that production incident with our important fintech client? That was different. As I watched my team burn out during those three days of firefighting, due to a misconfigured database, I realized something had to change.</p><p>Here’s the hard truth: The cloud was supposed to simplify things. Kubernetes was meant to make microservice management easier. Instead, we got more complexity, more toil, and more sleepless nights.</p><p>I’ve seen this pattern destroy teams across companies. I’ve watched brilliant engineers, engineering leaders and CISOs crumble under the weight of infrastructure chaos.<br> — <br>That’s why I built Mantis.</p><p>It’s not just another tool — it’s a complete reimagining of infrastructure configuration. One unified AI-powered engine replacing both Terraform and Helm. Modular configurations that actually work. Built-in validations that prevent disasters before they happen.</p><p>No more three-day firefights. No more burnt-out teams. No more infrastructure nightmares.</p><p>Want to break free from infrastructure chaos?</p><p>Try Mantis <a href="https://github.com/pranil-augur/mantis">https://github.com/pranil-augur/mantis</a></p><p>Join our discord community <a href="https://discord.gg/BJ8nKSmH">https://discord.gg/BJ8nKSmH</a> where I personally respond to every message.</p><p>Website: <a href="https://getmantis.ai/">https://getmantis.ai</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f8794b424b7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[2022 job search: Observations on engineering challenges]]></title>
            <link>https://pranildasika.medium.com/2022-job-search-observations-on-engineering-challenges-1b8910f6d3c7?source=rss-3b4516808b00------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b8910f6d3c7</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[growth]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[multicloud-management]]></category>
            <category><![CDATA[engineering-mangement]]></category>
            <dc:creator><![CDATA[Pranil Dasika]]></dc:creator>
            <pubDate>Wed, 26 Jan 2022 01:42:52 GMT</pubDate>
            <atom:updated>2022-02-02T20:53:57.367Z</atom:updated>
            <content:encoded><![CDATA[<p>After leaving my previous job to take a break for a few weeks in December, I started interacting with companies of different sizes to find my next job. I’m sharing my distilled notes speaking with several engineering leaders, and founders of companies, learning about some of the challenges and problems their teams are facing.</p><p>To put this in context, I was targeting fast-growing late-stage/public companies but also selectively explored a few very exciting early-stage startups. My sample size is really small (&lt;20 companies) and so these observations are by no means a global representation of the industry trends, but would love to share notes, and get feedback from others in the engineering community.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aG_tu56xthJ2Os9U.png" /><figcaption>I think modern engineering teams are like the Octanauts (from my daughter’s favorite Netflix show), who operate from their home base called Octopod. Different specialists come together on various rescue missions of other sea creatures.</figcaption></figure><h4>Need for delightful product experiences</h4><p>User time is precious. Apps cannot afford to become bottlenecks. Enterprise apps are seriously raising the bar for UX which has traditionally been a focus for consumer apps. Product-led growth (PLG) is a trending pattern in enterprise startups, to complement traditional sales-led growth. Onboarding, integration, and ongoing usage are super important for the product to win on its own.</p><h4>Need for speed</h4><p>The ability to ship fast is super important for customer-facing applications. Cycle times are fast reducing i.e time taken for developed features, bugs fixed to reach end-users. Both by design and necessity. Business demands, innovation, and competitive pressures + high severity security fixes and high impact bugs. For e.g Log4jShell touched every layer of the stack much as <a href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=cve-2014-0160">heartbleed</a> did a few years ago, and teams scrambled to roll out patches.</p><h4>Need for quality</h4><p>Balancing quality with velocity and tech debt. Even the best teams are struggling to resolve tech debt effectively. Was super excited to talk to a startup that’s trying to solve this through automated test generation that can be embedded right into the CI/CD pipeline. (DM me if interested, this is a very strong team with very ambitious goals).</p><h4>Need for data tracing</h4><p>Keeping production pipelines up and running reliably is hard. Identifying problems, esp data quality/correctness is harder. Debugging and resolving them is the hardest. Data cataloging, lineage tracking, and data validation tools are finding their way into the stack.</p><h4>Need for real-time processing</h4><p>Speed is paramount for automated, software-delivered applications. From medicine delivery to car-hailing to payments to monitoring — speed of data delivery and real-time analytics are common themes. Kappa architecture (Apache Flink), and real-time databases like Apache Pinot, Clickhouse, Druid came up several times.</p><h4>Need for standardized formats and APIs</h4><p>Startups were in different domains — cybersecurity, life sciences, enterprise sales enablement, and e-commerce fulfillment and were building data integration hubs that can talk to several upstream and downstream systems/applications. Deep domain expertise is needed to design optimal APIs — balancing flexibility, composability, customizability, and reusability.</p><p>One of the engineering leaders drew the parallel to the <a href="https://www.iab.com/wp-content/uploads/2016/03/OpenRTB-API-Specification-Version-2-5-FINAL.pdf">OpenRTB standard</a> in ad-tech, something that my team integrated while building an ads mediation platform at a previous startup.</p><h4>Need for multi-cloud</h4><p>The last two enterprise companies I worked at had to offer the product on several cloud providers, primarily AWS. Multi-cloud is now real, even for consumer applications. And not just for vendor lock-in concerns. Data residency, performance, latency, cost, disaster recovery, and sometimes to support bursty peak traffic that a single cloud provider may not support.</p><h4>Need for better cost management</h4><p>Public clouds made infrastructure easily accessible. They also increased the complexity and costs to operate different environments: dev, prod, staging, testing, performance, etc. Teams using different services across cloud providers and even multiple accounts and regions within the same provider make it difficult to track and manage costs in an optimal manner.</p><h4>Need for continuous delivery</h4><p>Continuous delivery was mostly limited to consumer applications. Enterprise and SaaS companies are catching up, realizing the risks of accumulating too many changes into large releases. Daily releases may not be always practical for many teams and products, but the requirement to deliver a software update on short notice is growing (Log4jShell!!).</p><p>Also, GitOps is now a common practice. Infra as code (IaC) is rapidly growing. Large teams still figuring out good patterns/models to use as teams and components grow. Migration from legacy scripts/systems is another problem teams are facing.</p><h4>Need for security/privacy</h4><p>Cloud makes services accessible — this is good. Cloud also makes data (and infra) very easily accessible — this is good too, but also bad if unprotected. Security is needed at every layer of the stack, protecting organizations from both internal and external threats.</p><p>I’ve had the opportunity to interview with some very impressive early and growth-stage startups and was left amazed by the innovation happening to empower developers to stay on top of security i.e shift security and privacy left into the Dev/CI/CD phase.</p><p>Another very exciting startup uses AI/ML to help prevent cybercrime in a modern multi-cloud world. (DM me if interested in engineering roles at these companies. Happy to make intros to some very inspiring founders!).</p><h4>Need for talent</h4><p>Companies of all sizes have a lot of complex problems than the talent needed to solve them effectively and efficiently. Hiring, retention, alignment, and executing against these ideas still remains the biggest challenge for teams.</p><h4>Thoughts?</h4><p>Would love to hear more about some of the challenges that your engineering team is facing, and how you’re addressing them. So, please comment if you have any thoughts.</p><p>Cheers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b8910f6d3c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ML landscape overview]]></title>
            <link>https://medium.com/data-science/ml-landscape-overview-8a9e48a28641?source=rss-3b4516808b00------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a9e48a28641</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pranil Dasika]]></dc:creator>
            <pubDate>Sat, 28 Nov 2020 05:45:49 GMT</pubDate>
            <atom:updated>2021-02-01T01:04:31.460Z</atom:updated>
            <content:encoded><![CDATA[<h3>ML Platform landscape overview</h3><h4>A birds-eye view into the key components of a modern enterprise ML Platform</h4><p>A query from an investor friend nudged me into a deep dive into the ML Platform landscape over the thanksgiving weekend. As I was gathering notes, I thought it’d be good to share them for the benefit of folks interested in understanding the space.</p><figure><img alt="Machine Learning Platform Architecture" src="https://cdn-images-1.medium.com/max/1024/1*Ni1g7F3ZVPYOabK0vBs-5Q.jpeg" /><figcaption>ML Platform Services (Image by Author)</figcaption></figure><p>I was particularly interested in the different components that make up a modern ML platform for a company, and I looked at 3 different classes of solutions:</p><ul><li>Both OSS and closed source solutions primarily built for internal consumption at hi-tech companies like Uber, Airbnb, Lyft, DoorDash.</li><li>Commercial startups I’m aware, of that offer these services as enterprise-grade software solutions. By no means is this meant to be an exhaustive list.</li><li>Commercial managed services from the 3 big cloud vendors, AWS/Azure/ Google Cloud.</li></ul><p>I identified the following key services, spanning the exploration, build, train, test &amp; production stages of the model development lifecycle (MDLC):</p><ul><li>Managed Notebook service</li><li>Model training service</li><li>Data labeling service</li><li>Workflow orchestration &amp; Data pipelines</li><li>Feature-stores</li><li>Model management services</li><li>Model serving/deployment service</li><li>Model monitoring and explainability service</li></ul><h3>Managed Notebook service</h3><p>Jupyter notebooks are the most popular IDE for data scientists. Besides their browser-based UI, Notebooks offer a great deal of flexibility for their day to day tasks, which could involve feature exploration, feature engineering, data wrangling, model building, and data analysis. Additionally, built-in visualization tools, ability to save and share as simple HTML files, make Jupyter notebooks ubiquitous in the Data Science world.</p><p>Specifically, in a company setting, when run as a managed notebook service, it could be offered with some additional utilities over the vanilla laptop run notebook service, and this could drive significant efficiency for a decently big team of data scientists. For e.g:</p><ol><li>Pre-built environments with commonly used ML libraries and easy access to algorithms.</li><li>Built-in connectivity to access-protected data-warehouses, feature stores</li><li>Built-in support for cloud backed GPU based model runtimes, which developer laptops cannot offer.</li><li>Ability to spin up training jobs from within notebooks.</li></ol><p><strong>OSS/Well known frameworks</strong>: Airbnb Redspot service, Uber DSW, Kubeflow</p><p><strong>Enterprise Startups</strong>: Determined AI, Databricks, DominoData labs, Paperspace Gradient</p><p><strong>Cloud Vendors</strong>: AWS Sagemaker</p><h3>Model Training</h3><p>Model training jobs are iterative, stateful, incremental, and compute-intensive — requiring distributed computing for speedy turnarounds. To iterate fast on model development, data scientists need to be able to spin up training jobs without having to jump through the hoops of hardware setup, working with their IT team.</p><p>In addition, smart ML training services provide access to automated hyper-parameter search algorithms, which allow for efficient and optimized training convergence. They also offer built-in resiliency via checkpoint saving and restoring. The ability to provision CPU/GPU resources for distributed training algorithms, and support for multiple popular frameworks, are also key characteristics of an enterprise-wide training platform.</p><p><strong>OSS/Well known frameworks</strong>: Airbnb BiqQueue service, Uber Horovid, Kubeflow Fairness, TFX.</p><p><strong>Enterprise Startups</strong>: Determined AI, Databricks, DominoData labs</p><p><strong>Cloud Vendors</strong>: AWS Sagemaker, Azure ML</p><p><strong>References</strong>:</p><p><a href="https://determined.ai/blog/determined-ai-sagemaker-comparison/">https://determined.ai/blog/determined-ai-sagemaker-comparison/</a></p><h3>Data labeling</h3><p>Data powers ML applications. High-quality labeled training data is imperative for the high-performance of models. Labeling data in a scalable reliable way requires specifically designed workflows and expertise. Smart and intuitively designed labeling workflows can bring in significant efficiency gains while dealing with a large labeling workforce on large data projects.</p><p><strong>Enterprise Startups</strong>: Scale AI, Snorkel, LabelBox</p><p><strong>Cloud Vendors</strong>: AWS Sagemaker ground truth</p><h3>Workflow orchestration and Data pipelines</h3><p>Machine learning production workflows involve data movement between multiple sub-systems at data ingestion, preparation, training, and scoring time. Data needs to be moved, massaged into the desired format before feeding into models for training and inference.</p><p>Various teams and products inside an organization use different locations to store data that may be consumed in a completely different part of the organization, which could be in a different data center or cloud or even geography. Workflow orchestration systems allow for composing complex workflows, scheduling, and executing them reliably at scale in production.</p><p>Modern workflow services build on top of Kubernetes and focus on building expressive ways to represent and execute complex DAG style workflows while offloading the aspects of compute-scaleout to the underlying k8s system.</p><p><strong>OSS/ Well known frameworks: </strong>Flyte, Airflow, Argos, Kubeflow pipelines</p><p><strong>Enterprise Startups</strong>: Prefect, Pachyderm, Databricks, Domino Data</p><p><strong>Cloud Vendors</strong>: AWS Data Pipeline, Google AI pipelines(Kubeflow pipelines), Azure ML Pipelines</p><p><strong>References</strong>: <a href="https://www.youtube.com/watch?v=oXPgX7G_eow&amp;t=1152s">https://www.youtube.com/watch?v=oXPgX7G_eow&amp;t=1152s</a></p><p><a href="https://kapernikov.com/a-comparison-of-data-processing-frameworks/">https://kapernikov.com/a-comparison-of-data-processing-frameworks/</a></p><h3>Feature-stores</h3><p>Feature handling is a fairly involved process, both during model development and at production time — feature discovery, exploration, extraction, transformations, and serving. Repeatability and reliability of these operations at scale, and serving features to multiple ML applications for scoring, version tracking and maintaining data lineage, and metadata are key responsibilities handled by <em>feature stores</em>. Additionally, feature stores also make backtesting of models easier, by backfilling historical data when new features are added.</p><p><strong>OSS/ Well known frameworks: </strong>Airbnb Zipline, Uber’s Michaelangelo Palette, Facebook’s FBLearner, <a href="https://www.kubeflow.org/docs/components/feature-store/overview/">Kubeflow</a></p><p><strong>Enterprise Startups</strong>: Tecton, Scribble, Hopsworks from LogicalClocks</p><p><strong>References</strong>:</p><p><a href="https://www.tecton.ai/blog/what-is-a-feature-store/">https://www.tecton.ai/blog/what-is-a-feature-store/</a></p><p><a href="https://www.scribbledata.io/resources-feature-store-guide">https://www.scribbledata.io/resources-feature-store-guide</a></p><p><a href="https://hackernoon.com/the-essential-architectures-for-every-data-scientist-and-big-data-engineer-f21u3e5c">https://hackernoon.com/the-essential-architectures-for-every-data-scientist-and-big-data-engineer-f21u3e5c</a></p><h3>Model management and experiment management</h3><p>Once models are trained, they can be cataloged and tracked using model management systems.</p><p>Data Scientists also have to work with Business/ Product teams to run multiple experiments. Large teams would need to track changes to models during experiments in a scalable way, which involves capturing things like code versions, data versions, hyperparameters, environment, and metrics.</p><p>The key challenge lies in supporting popular model frameworks, like XGBoost, TensorFlow, Keras, Pytorch, Scikit Learn, LightGBM.</p><p><strong>OSS/ Well known frameworks: </strong>MLFlow, TensorBoard</p><p><strong>Enterprise Startups</strong>: Weights and Biases, Neptune, Comet ML, Paperspace Gradient</p><p><strong>References</strong>:</p><p><a href="https://towardsdatascience.com/tracking-ml-experiments-using-mlflow-7910197091bb">https://towardsdatascience.com/tracking-ml-experiments-using-mlflow-7910197091bb</a></p><p><a href="https://neptune.ai/blog/experiment-management">https://neptune.ai/blog/experiment-management</a></p><h3>Model serving/deployment</h3><p>Once models are trained and need to be deployed either to serve as part of live applications or for offline scoring, you need a consistent and reliable mechanism within the company. Model serving and deployment requires a few different considerations:</p><ol><li>CPU/GPU/TPU based environments</li><li>Support for composable, stacked models</li><li>Model dependency management</li></ol><p>Many enterprises, fresh from their digital transformation journeys, recognize the need for standardized deployment formats across teams to drive efficiencies. As a result, containerized model formats are gaining popularity as Kubernetes can be used to serve them up at scale while hiding the complexity of scaling, memory, and CPU utilization.</p><p><strong>OSS/ Well known frameworks: </strong>KF Serving, TF Serving, Seldon core</p><p><strong>Enterprise Startups</strong>: Algorithmia, Seldon, AnyScale, Paperspace Gradient</p><p><strong>Cloud Vendors</strong>: AWS Sagemaker</p><p><strong>References</strong>:</p><p><a href="https://www.youtube.com/watch?v=-On2Ty259yo">https://www.youtube.com/watch?v=-On2Ty259yo</a></p><p><a href="https://huyenchip.com/2020/06/22/mlops.html">https://huyenchip.com/2020/06/22/mlops.html</a></p><h3>Model monitoring and explainability</h3><p>Once models are deployed in production, keeping an eye on their performance is key to ensure you get the ROI for all the hard work the teams have put in to reach your business goals. The basic premise of machine learning, which is to train and come up with a generalized model for unseen data can actually be validated through constant monitoring in production.</p><p>Also, model training is not a one-time process. Models need to be updated and re-trained over time. Model monitoring solutions mainly look for bias, drift, outliers, data quality issues, and also overall performance of the model over time, which inform retraining decisions. Model monitoring solutions also allow for comparison of challenger-champion models and help speed up the model promotion process.</p><p>Model explainability takes a step further into inspecting why the model makes a decision. Techniques like LIME and SHAP, work well for simpler models, but for more advanced insights for deep learning models, techniques like Integrated Gradients offer deeper visibility into the attributions of the different features, and their interactions, especially for deep learning models.</p><p><strong>OSS/ Well known frameworks (Explainability): </strong>SHAP, LIME, Captum</p><p><strong>Enterprise Startups</strong>: Fiddler, Arthur, Arize</p><p><strong>Cloud Vendors</strong>: AWS Sagemaker</p><p><strong>References</strong>: <a href="https://blog.fiddler.ai/2020/04/explainable-monitoring-stop-flying-blind-and-monitor-your-ai/">https://blog.fiddler.ai/2020/04/explainable-monitoring-stop-flying-blind-and-monitor-your-ai/</a></p><p>If you think there’s any other key component that didn&#39;t make it to the list, please leave a comment, so that I can follow up!</p><p><strong>*Update 1 on 12/7/20</strong>: AWS announced 3 new Sagemaker services during the AWS re:Invent on 12/1/20</p><ol><li><a href="https://aws.amazon.com/sagemaker/feature-store/">AWS Sagemaker feature store</a></li><li><a href="https://aws.amazon.com/sagemaker/data-wrangler/">AWS Sagemaker data wrangler</a></li><li><a href="https://aws.amazon.com/sagemaker/pipelines/">AWS Sagemaker pipelines</a>, which is different from the AWS Data pipeline mentioned above.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a9e48a28641" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/ml-landscape-overview-8a9e48a28641">ML landscape overview</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>