<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tony Fontana on Medium]]></title>
        <description><![CDATA[Stories by Tony Fontana on Medium]]></description>
        <link>https://medium.com/@tfohio96?source=rss-a65e9be95d2c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Yss1lgTeTz9mxd5FdTB3dQ.png</url>
            <title>Stories by Tony Fontana on Medium</title>
            <link>https://medium.com/@tfohio96?source=rss-a65e9be95d2c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:45:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tfohio96/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What Open Source Summit Europe Taught Us About Agents and Memory]]></title>
            <link>https://medium.com/99p-labs/what-open-source-summit-europe-taught-us-about-agents-and-memory-e9edac18c454?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9edac18c454</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Tue, 13 Oct 2026 20:15:30 GMT</pubDate>
            <atom:updated>2026-10-13T20:15:30.423Z</atom:updated>
            <content:encoded><![CDATA[<p>Written by Tony Fontana and Brian Coy</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*k3Fzo2GAIMQ144M9klXHuQ.png" /></figure><p>Last week we went to Prague for the Linux Foundation’s Open Source Summit Europe and Embedded Linux Conference. We went to learn what the industry and open source community are doing for agentic transformation, and to see where the cutting edge is with open source tools and local models. That part matters a lot for us. At 99P Labs our team is working on an AI transformation project for software development, and we run our own models on our own hardware. We wanted to know what other teams are building and if they’re running into the same problems we are. Turns out, a lot of them are.</p><p>The theme that kept coming up was context. Not just how big the context window is, but how teams organize what an agent knows over time. One talk from AWS on multi-agent systems put it simply: the model is stateless, and a context window is not memory. When an agent “forgets” something, it’s usually missing state, not hallucinating. They split memory into short term memory for the current task and session, long term memory for what was learned over time, and episodic memory for summaries of past tasks and the decisions that were made.</p><p>That lined up almost exactly with how our team has been building our knowledge layer. We think about it in levels. A user’s session holds the current context. When decisions get made or a task wraps up, the summaries move up to team memory. What the team learns gets distilled and moves up again into organizational memory, where documentation and lessons live across teams. Next to all of that we keep a code graph of how the code itself connects. Hearing other teams land on the same shape was a good sign we’re on the right track.</p><p>The other big learning was about RAG. A lot of companies start their AI journey there. You put your documents in a vector database, someone asks a question, and the system pulls back the closest chunks of text. That’s useful, but RAG is just search. It finds text that looks similar to your question. It can’t tell you why a decision was made, it can’t connect the answer to the code you’re working on, and it definitely can’t go fix the problem.</p><p>Foundation Capital has a good write up on this called <a href="https://foundationcapital.com/ideas/context-graphs-ais-trillion-dollar-opportunity">Context Graphs: AI’s Trillion-Dollar Opportunity</a>. Their point is that most systems store what happened, but not why. The why only gets captured if something is in the loop while the work is happening. A search index sitting off to the side never sees it.</p><p>This is where the conference gave us real backing for something our team has been saying for the past year. No single piece does it on its own. The model does the reasoning. RAG and our knowledge graphs hold the knowledge. The harness is everything around the model, like the tools, MCP connections, and agents that decide what to do next. With the harness in place, an agent doesn’t just search once and hand back text. It figures out what it needs to look up, pulls the right context from memory and the code graph, gives a detailed answer that fits the actual problem, and then applies the solution. It writes the code, runs the checks, and saves what it learned back into memory so the next person doesn’t start from zero. That loop is what turns a search box into a software factory.</p><p>Outside of memory, we caught a bunch of other talks worth mentioning. A couple were close to home, like Neo4j’s take on GraphRAG as a knowledge layer for enterprise AI and Dynatrace benchmarking open source agent memory on Kubernetes with OpenTelemetry. Security and trust came up a lot too: sandboxing agents with Kata and microVMs, permissioned agents that can run offline, identity for agent-to-agent communication, and an open benchmark measuring how secure the code from coding agents really is. There was also plenty on running agents in production on Kubernetes, observability for tracing what an agent actually did, MCP server design, and getting real work out of small language models and CPU-only setups. And on the embedded side, talks on drones landing on high voltage power lines and a flight simulator that runs inside the autopilot were a fun change of pace.</p><p>The big takeaway from Prague is that the community is converging on the same ideas. Agents need memory with structure, from short term all the way up to organizational. Search on its own isn’t enough, because the value comes from agents and tools that can use that knowledge in context and act on it. And the teams getting the most out of AI aren’t just picking a better model, they’re building the harness and memory around it.</p><p><em>Follow 99P Labs on Medium for more updates on our research.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9edac18c454" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/what-open-source-summit-europe-taught-us-about-agents-and-memory-e9edac18c454">What Open Source Summit Europe Taught Us About Agents and Memory</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nvidia GTC 2026: From SaaS to Agents as a Service]]></title>
            <link>https://medium.com/99p-labs/nvidia-gtc-2026-from-saas-to-agents-as-a-service-398be523c0ba?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/398be523c0ba</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Mon, 23 Mar 2026 14:38:15 GMT</pubDate>
            <atom:updated>2026-03-23T14:38:50.094Z</atom:updated>
            <content:encoded><![CDATA[<p>Written by: Tony Fontana</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ivrxsm7aR82JbSCXW_yZIg.jpeg" /></figure><p>GTC is big. Like, genuinely overwhelming. Thousands of people, a three hour keynote at an arena, and more announcements than you can process in a week. I went in knowing there would be a lot of noise and trying to focus on what was actually relevant to what we are building at 99P Labs.</p><p>Our team is working on agentic software development systems. The short version is: we are building tools and platforms where LLMs work together in automated workflows to handle development tasks, testing, requirements gathering, and knowledge storage. We run experiments, deploy open source models, and make real architecture decisions constantly. GTC felt like a good place to check our thinking against what the rest of the industry is doing.</p><p>Here is what stuck with me.</p><h3>There is real signal here, but you have to look for it</h3><p>I want to be honest about this. GTC is one of the most hyped conferences in tech right now and Jensen Huang’s keynote is a masterclass in making everything sound like a turning point in history. Some of it is. Some of it is a product announcement dressed up as a paradigm shift.</p><p>The sessions and hallway conversations are where the useful stuff lives. That is where you find engineers who are actually building things and running into the same problems you are. The keynote sets direction. The deeper talks are where you find the detail. Go with specific questions and use those as your filter.</p><h3>OpenClaw as an OS for agents</h3><p>One of the bigger themes at GTC was NVIDIA’s take on OpenClaw, which they wrapped with an enterprise security layer called NemoClaw. Jensen compared it to Linux and Kubernetes on stage, which sounds like hype, but the underlying point is worth thinking about.</p><p>The idea is that traditional operating systems manage resources for humans using software. What OpenClaw is pushing toward is an OS layer for agents, software that manages permissions, sandboxing, and resource access for AI doing the actual work. NemoClaw adds a security runtime called OpenShell on top of that, policy-based guardrails, a privacy router, and local model support. It installs in a single command. It is alpha stage and rough around the edges by NVIDIA’s own admission, but the direction is interesting.</p><p>For what we are building, this framing is useful. We are actively working through how agents in our system get access to tools, how they stay constrained, and how they communicate with each other. Watching how NemoClaw handles that infrastructure layer gives us a useful reference even if we are not deploying it directly.</p><h3>The SaaS to agents shift is real and already happening</h3><p>Jensen said on stage that every SaaS company will become an AGaaS company, agents as a service. It landed as a quotable line but it also reflects something you could see across the conference. Companies are not just adding AI features. They are redesigning products around agents doing the work instead of humans clicking through interfaces.</p><p>For us this just reinforces the direction we are already headed. The platforms and tools we build on top of are shifting toward agentic architectures, and we need our infrastructure to be ready for that before it becomes urgent.</p><h3>Specialized models matter more than I expected</h3><p>One of the more practically useful sessions I attended was on SWE-grep from Silas Alberti at Cognition. SWE-grep is a model they trained specifically for fast context retrieval in coding agent workflows. Before it existed, their agents were spending over 60% of first turn time just searching for relevant code, 20 to 60 seconds, before making a single edit. SWE-grep gets that down to under 3 seconds using reinforcement learning and parallel search calls.</p><p>The bigger takeaway for me was not about grep specifically. It was about where task-specific models actually pay off in a multi-agent system. We are designing workflows where different agents handle different parts of the development pipeline, and there is a real question of when you use a large general model versus a smaller specialized one. The answer is not always the biggest model. Sometimes a fast specialized model at the right step speeds up the whole system significantly without sacrificing quality. That tradeoff is worth taking seriously.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jnMGrFr_6MP0ylIC6fSm1g.jpeg" /></figure><h3>Proving your agents are not making major mistakes</h3><p>This did not get much stage time but it kept coming up in sessions and conversations. When you deploy AI coding agents that are taking real actions in a codebase, writing code, running tests, opening pull requests, how do you actually verify they are not doing something wrong?</p><p>There is a recent paper called Agent Behavioral Contracts that proposes bringing design by contract principles to agentic systems. Traditional software has type systems, assertions, and APIs to catch incorrect behavior. AI agents operating on prompts have none of that unless you build it. The paper proposes a formal framework for specifying and enforcing agent behavior at runtime.</p><p>This is a near term concern for us. We are building toward agents taking real actions and the contract layer, how you constrain and verify what they do, is something we are actively thinking through. Worth reading if you are in a similar place.</p><h3>NVIDIA is as much a software company now as a hardware one</h3><p>The meta-signal from GTC this year was that NVIDIA is stacking serious software on top of their hardware. NemoClaw, cuVS, Dynamo, the Nemotron model ecosystem, the Agent Toolkit. They are building the infrastructure layer for agentic AI and tying it to their stack in ways that will be hard to move away from once adopted.</p><p>For builders, this means there is a growing set of infrastructure to plug into rather than build from scratch. It also means being intentional about which parts of that stack are genuinely useful to you versus which ones are just gravity toward NVIDIA’s ecosystem. Both can be true at the same time.</p><h3>The access to NVIDIA engineers was genuinely useful</h3><p>This surprised me. We got time directly with NVIDIA engineers onsite to work through specific technical problems we are hitting in our current deployments. Not a demo or a sales conversation. Actual troubleshooting with people who knew the stack deeply and were willing to spend time on our problem. That kind of access at a conference this large is not something I expected. For anyone going next year with a specific technical problem, take advantage of it.</p><h3>Worth going?</h3><p>If you are working on agentic AI systems, yes. Especially if your questions are around deploying coding agents safely, designing multi-agent workflows, benchmarking open source models, or thinking through knowledge retrieval at scale. Those conversations are happening at GTC and you will find people who are further along than you and people who are asking the same questions.</p><p>Go with specific questions, be skeptical of the keynote framing, and find the sessions where people are showing real systems with real numbers. The signal is there if that is what you are looking for.</p><p>Follow 99P Labs on <a href="https://medium.com/99p-labs"><em>Medium</em></a><em> for more updates on our research, and connect with us on </em><a href="https://www.linkedin.com/company/99p-labs"><em>LinkedIn</em></a><em>. Want to talk about what we are building? Reach us at research@99plabs.com.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=398be523c0ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/nvidia-gtc-2026-from-saas-to-agents-as-a-service-398be523c0ba">Nvidia GTC 2026: From SaaS to Agents as a Service</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Emulating the Future: Why 99P Labs Is Building a Cloud-Edge Emulator]]></title>
            <link>https://medium.com/99p-labs/emulating-the-future-why-99p-labs-is-building-a-cloud-edge-emulator-0389fc28d56f?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/0389fc28d56f</guid>
            <category><![CDATA[emulator]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Mon, 05 May 2025 15:00:58 GMT</pubDate>
            <atom:updated>2025-05-05T15:00:58.449Z</atom:updated>
            <content:encoded><![CDATA[<p>Written By: Tony Fontana</p><p>At 99P Labs, we’re committed to building the future of sustainable mobility, intelligent infrastructure, and edge-native applications. As these systems grow in complexity — spanning cloud data centers, edge servers, mobile devices, and real-world environments — testing them becomes a major bottleneck. We need a new way to design, validate, and iterate. That’s where <strong>emulation</strong> comes in.</p><p><strong>The Problem: Complex Systems, No Real Way to Test</strong></p><p>Modern applications like autonomous vehicles and smart cities rely on dynamic cloud-edge systems. These systems must balance latency, bandwidth, compute resources, and privacy in real time. Traditional testing methods are slow, expensive, and risky. Worse, there are no standard benchmarks to evaluate how these systems will behave in real-world conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*aqLPQroWkX4dIyEvKtbbCw.jpeg" /><figcaption>Cloud-Edge Continuum</figcaption></figure><p><strong>The Solution: Cloud-Edge Emulation</strong></p><p>A cloud-edge emulator gives us a safe, controllable environment to simulate and test how distributed systems perform under different scenarios. With it, we can:</p><ul><li><strong>Optimize performance</strong> under variable compute and network conditions.</li><li><strong>Validate system scalability</strong> without real-world infrastructure.</li><li><strong>Accelerate development cycles</strong> by catching issues early.</li><li><strong>Test new ideas</strong> safely before hardware deployment.</li><li><strong>Run trade-off studies</strong> (e.g., cloud vs. edge vs. device processing).</li></ul><p><strong>What We’re Emulating</strong></p><p>To accurately model future systems, we’re building an emulator that covers three core areas:</p><p><strong>1. Compute Emulation</strong></p><p>We simulate compute at all levels — cloud, edge, and device. This includes:</p><ul><li>Different hardware types and resource limits</li><li>Power constraints and failure scenarios (e.g., nodes dropping offline)</li><li>AI/ML model deployment and workload stress testing</li></ul><p><strong>2. Network Emulation</strong></p><p>We recreate real-world networking challenges:</p><ul><li>Simulated latency, jitter, and packet loss over 5G, WiFi, and satellite</li><li>Network dropouts and recoveries</li><li>Effects of physical distance and device mobility</li></ul><p>The goal is a <strong>digital twin of the real network environment</strong>.</p><p><strong>3. Physical and Real-World Conditions</strong></p><p>We introduce real-world disruptions like:</p><ul><li>Edge nodes going offline</li><li>Environmental interference (weather, power outages)</li><li>Distance between devices and edge infrastructure</li><li>Multi-modal interactions in smart cities (e.g., vehicles, drones, IoT)</li></ul><p>Together, these enable us to understand how systems fail, adapt, and scale in the real world.</p><p><strong>MVP: Building the First Prototype</strong></p><p>To validate this approach, we’re developing an internal <strong>minimum viable emulator</strong> using:</p><ul><li><strong>Dockerized compute pods</strong> that simulate distributed nodes (CPU, GPU, memory partitioning)</li><li><strong>Virtual networks</strong> that throttle bandwidth, inject latency, and simulate distance</li><li><strong>Application deployment</strong> across pods to test edge AI workloads</li></ul><p>This small-scale system allows us to emulate edge scenarios without needing physical infrastructure.</p><p>In parallel, we’re researching more advanced networking models to simulate moving devices — such as vehicles changing location or going in and out of range. This modular approach allows us to test ideas in isolation before combining them into a full-system emulator.</p><p><strong>Why Emulation Matters to 99P Labs</strong></p><p>This emulator directly supports our research priorities:</p><ul><li><strong>Sustainable mobility</strong>: Simulate traffic and vehicle behavior to design congestion-aware, low-latency systems.</li><li><strong>Intelligent infrastructure</strong>: Test energy and sensor systems under realistic stress.</li><li><strong>Faster innovation</strong>: Enable students and researchers to experiment, break, and learn — without needing full physical deployments.</li></ul><p>Please consider following us on Medium to follow the progress of this research project!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0389fc28d56f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/emulating-the-future-why-99p-labs-is-building-a-cloud-edge-emulator-0389fc28d56f">Emulating the Future: Why 99P Labs Is Building a Cloud-Edge Emulator</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enhancing Developer Experience and Machine Learning Workflows with MLOps]]></title>
            <link>https://medium.com/99p-labs/enhancing-developer-experience-and-machine-learning-workflows-with-mlops-d00b24773c86?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/d00b24773c86</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[kubeflow]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Fri, 22 Mar 2024 17:10:31 GMT</pubDate>
            <atom:updated>2024-03-22T17:10:31.907Z</atom:updated>
            <content:encoded><![CDATA[<p>Written by: Tony Fontana and Luka Brkljacic</p><p>Are you curious about the practical application of MLOps? Or perhaps you’re interested in learning about the day-to-day experiences of a machine learning engineer? This blog delves into the current state of our machine learning platform and our team’s adoption of MLOps practices. Whether you’re a seasoned ML professional or just starting your journey, you’re sure to find valuable insights in the content that follows.</p><p>At 99P Labs we have been supporting data scientists and software engineers with a research platform since our team’s inception. Our goal of software research has been supported by a community of university partners and internal collaborators. To make working together simple and efficient, and to smooth onboarding we have offered a developer portal and data science environment for our developers and scientists. Over the years we have deployed many versions of these platforms, if you want to read about one of our previous versions please you can check out <a href="https://medium.com/99p-labs/enabling-data-science-teams-of-any-size-43c8d8029b0">this blog</a>. We also have a blog describe our developer and data community that you can <a href="https://medium.com/99p-labs/99gpt-building-a-chatbot-fdde8b689df4">read here</a>.</p><p>In the earlier version of our data science environment, the core aim was to bring together all our developers and data scientists onto our platform. They had the freedom to develop Python code on their individual machines, subsequently submitting Spark jobs to our cluster to distribute workloads. However, this resulted in each developer managing their own coding environment, which proved to be hurtful to the overall user experience. Not only was the interface poor, but it also discouraged collaborative efforts among team members.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*Ld32d1N3FjqvlXD7sOhGaw.png" /></figure><p>The benefit of this system is that the data scientists could submit their heavy workloads to a cluster that had much more capability than a user’s local machine, meaning the data scientists were able to generate heavy models and run big jobs. One drawback of this was converting python jobs into spark jobs (which is java based). This translation was an inefficient step, and we wanted to upgrade in our next data science platform.</p><p>We also had increased our internal team capability and wanted to start generating more large machine learning models and LLMs for our research projects. Our company, Honda Research Institute US also holds strong capability and ML engineers across the company. We wanted to increase collaboration across the company and within our own team of ML engineers and data scientists. Our team wanted to take our DevOps first mindset when building a platform and adopt this into a MLOps mindset. This is focused on creating a strong platform to help enable machine learning engineers first. You can read more about having the MLOps mindset in a blog I wrote previously found here.</p><p>Our new developer experience and ML platform is supported by Kubeflow. An open source toolkit that allows data scientists and ML engineers to collaborate, experiment, deploy their own code and serve ML models. Kubeflow is containerized and runs in Kubernetes, which makes it a perfect fit into our existing platform structure. This helps reduce the workload for the platform (DevOps, MLOps) engineers and make the DS and ML engineers more self-reliant. The ML engineers can create their own data pipelines using python code and then deploy their code in a distributed way into containers in the Kubernetes cluster. The users come to the platform with their browser, code their scripts, and applications and use a python library to add pipeline capability. This then deploys containers on the backend that run the jobs and have output. The output is saved into persistent storage, which models and notebooks can access later. The model also gets served in its own container and has an API endpoint that can then be used for applications. This workflow is to run models in a production environment quickly, and securely. This user experience is a great step forward from our previous environment. Previously users could only write code locally, and run scripts in a cluster, but after there is no direct output besides the job completion. No persistent storage and no way to access a created model. This new iteration with Kubeflow provides an entire end to end development, research, and production platform for machine learning projects. Here are some of the features of Kubeflow that our ML engineers appreciate:</p><p>⦁ Experimentation tools<br>⦁ Workload scale <br>⦁ Versioning + collaboration tools<br>⦁ Storage access in the cluster<br>⦁ Self service running of code and pipelines<br>⦁ Self service model deployment<br>⦁ Production environment</p><p>Lets walk step by step through our new developer experience:</p><p>Besides having dynamic capability, we want our data scientists and ML engineers to have a good development experience when using anything on our platform. Lets walk through an overview on the developer experience and starting, creating, training, and deploying a model on Kubeflow:</p><ol><li>The machine learning engineers and data scientists reach Kubeflow in their browser, login and access their python notebooks. If they are starting a new project, they create a new notebook and can first start with experimenting on their dataset. They bring the dataset in by accessing it through s3 or minio storage. At the start of any project, they’re not usually sure about the exact approach they’ll take. That’s why it’s key to have a space for quick trial-and-error, it allows the engineers to be self sufficient and not need access to any platform engineers to get started and make mistakes and find solutions quick.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJ66cyVoLd5bNseatTYVdg.png" /></figure><p>2. Once the DS / ML engineer has a good understanding with the structure of their project, which libraries they are going to use, which methods for creating and training a model, then they can start to break everything into functions. These functions then can be transitioned into components of Kubeflow pipelines. Our ML engineers report that this process is a bit difficult and has a learning curve for the Infrastructure as Code library- kfp (Kubeflow pipelines). This is a library that allows the user to create data pipelines right from their Juptyer notebooks, with an Infrastructure as Code process. This process is currently being developed by the open source community and we are excited for kfp to improve and support more features, as it is a bit complex currently. This is probably the biggest complaint with our ML engineers, but after working with our MLOps engineer they were able to figure out how to make it work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w23RU76pv8NzbSD8auQIYg.png" /></figure><p>3. After developing the ML pipelines the user can run this process and this creates a pipeline object in Kubeflow. This pipelines then has a history of each run and each test of the pipeline. This is one of the main reasons and benefits of Kubeflow. This whole process does not require a platform engineer to deploy code, monitor the deployment, and make iterations. You can see which step in the pipeline failed or tune parameters without going back into the code. This means once the pipeline is created anyone with access could rerun this process with different parameters to fit their own needs. Each step in the pipeline gets deployed into its own container, which helps distribute the compute across your cluster. And you can run certain processes asynchronously to help speed up processes. This is much more efficient than running your code on a normal VM, as it allows for very large scale and distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VVWso2lH4RXpjL1Bl1iMFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eUoc_G12I64lbuXnYrYWLA.png" /></figure><p>4. After running the ML pipeline, next is serving the model using KServe. This deploys the model in Kubernetes, where it runs inside its own container and has an endpoint for applications to refer to a model. For example, if the model was trained for image classification, you could write an application that submits an image to the model and gets a classification and confidence level returned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XutXpR7rYOFY8ZvDRrpgTA.png" /></figure><p>This project is open source, which is exciting for us. Right now, we’re working closely with the Kubeflow community to report any issues and offer support. Even though we spot a lot of room for the project to grow and get better, we already find it really useful. Plus, it’s got a bunch of other features that we didn’t cover in this blog post, just to keep things short.</p><p>Our team outlined three use-cases for getting familiar with and testing the new platform: Simple Image Classification, Telematics Data Clustering &amp; Classification, 99P Blogs Dataset LLM Chatbot.</p><h3>Use-case 1: Simple Image Classification</h3><p>The first use-case uses a well-known image dataset, <a href="https://www.cs.toronto.edu/~kriz/cifar.html">CIFAR-10</a>, to create a simple image classifier using a Convolutional Neural Network (CNN). This dataset consists of 60,000 32x32 color images in 10 classes (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck), with 6,000 images per class. The goal for this use-case was use our CNN model to correctly classify new images based on the training data. For example, given a new picture of a dog, the goal is for our model to correctly label that image as being of a dog.</p><p>To accomplish the aforementioned goal, we converted the following high-level steps into a Kubeflow pipeline. Each of these steps will be explained in further detail below:<br>⦁ Loading the dataset<br>⦁ Pre-processing the data<br>⦁ Defining the CNN model<br>⦁ Training &amp; evaluating the data<br>⦁ Serving the model</p><p>The first step of the pipeline is very straightforward. We simply load the from <a href="https://keras.io/api/datasets/cifar10/">tensorflow’s keras datasets</a> and split the data into training and testing sets. We then save the data as a <a href="https://www.kubeflow.org/docs/components/pipelines/v2/data-types/artifacts/">kfp artifact</a> so that we can pass the datasets along to other pipeline components.</p><p>The next step in our pipeline then takes the output artifacts from step one as inputs. This data is then normalized, and some logging metrics are recorded. These metrics will be accessible from the Pipelines UI. Finally, we save the processed data as kfp artifacts.</p><p>The third step does not take in any input, but simply defines our CNN model and saves it as a kfp artifact to be passed along the pipeline.</p><p>The fourth step of the pipeline is where most of the actual work happens. This component takes in the CNN model, processed training and testing sets, and user-defined parameters as input. It then uses these inputs to compile, fit, and evaluate the model. As part of this process, two metrics are saved which can be viewed from the Pipelines UI: a confusion matrix and the model’s loss and accuracy. Finally, the trained model is saved as an artifact.</p><p>The final step in our pipeline takes the trained model as input and serves it using KServe. This creates an endpoint that can be queried with a simple JSON request, as well as viewed in the Endpoints UI.</p><h3>Use-case 2: Telematics Data Clustering &amp; Classification</h3><p>The second use-case built on the pipeline framework described in use-case 1. This time, we used synthetically generated telematics data (generated from real Honda driving data) to cluster drivers based on their driving behavior as defined by six unique driving-related features (speed, longitudinal acceleration, latitudinal acceleration, steering angular velocity, brake input pressure, acceleration pedal position). These clusters were then added to the data as labels. This labeled dataset was then split into training, testing, and validation sets and used to train a tensorflow classifier neural network. As in the first case, the trained model was served using KServe and queried with a simple JSON request.</p><h3>Use-case 3: 99P Blogs Dataset LLM Chatbot</h3><p>The 99P blogs chatbot was developed to be deployed on our platform, not as a Kubeflow pipeline. It is currently operational, and you can view it <a href="https://medium.com/99p-labs/99gpt-building-a-chatbot-fdde8b689df4">here.</a> Our intention with the third kfp use-case is to take this LLM application and transform it into a Kubeflow pipeline. This work has begun and uses the same basic framework described in the first two use-cases, however due to the complex nature of the LLM application and the fact that it was not built with kfp in mind it is going much more slowly than the first two use-cases. Still, we are confident that we will soon finish this final use-case and round out our first-round testing of Kubeflow.</p><h3>ML Engineer’s perspective on Kubeflow</h3><p>I came into Kubeflow with no prior knowledge or experience of the product. Given this, the initial learning curve was very steep and it took about a week of detailed digging through documentation, forums, and trail-and-error to finally get comfortable with Kubeflow pipelines (kfp) and the development workflow in general. However, once I became comfortable and everything “clicked”, the experience was fairly smooth overall.</p><p>First, let’s get into the pain points. The biggest troubles I had working with kfp had to do with things that will be fixed in the near future, so I will not go into great detailed about them. These included having to go through DevOps for debugging, issues related to kfp versioning, and no GPU access. Other than these issues, I didn’t experience any big roadblocks. There are definitely things that I wish could be different, such as how metrics and visuals are displayed in the Pipelines UI and the convoluted and proprietary Artifacts framework. These things made my development experience slightly more painful and longer, but ultimately did not block and movement toward our end goal. Next, let’s talk about what went well.</p><p>In my opinion, the biggest strength of Kubeflow is that everything is in one place. You can spin up a Jupyter notebook with as many resources as you need, do your development in that notebook, run and schedule pipelines, then view those pipelines and their individual components within the Pipelines UI. Once everything looks good, you can go back to your notebook, serve and query the model, and once again be able to view it in the Endpoints UI. The fact that you can switch between a coding view and a visual representation of your code is super helpful in the development process. Additionally, being able to organize projects using Experiments and Pipeline versions is a huge win as it allows for very quick and easy iterative development.</p><p>Overall, I think Kubeflow — and particularly kfp — can be a great tool given some initial time investment and patience. The updates that are coming soon should also greatly improve the user experience, increasing the value proposition for teams even more.</p><h3>Conclusion</h3><p>Our team’s experience with Kubeflow has been positive, and we are excited to continue leveraging the platform for their machine learning research and development efforts. The ability to create end-to-end ML pipelines, from data ingestion to model serving, has been a game-changer, empowering the data scientists and ML engineers to be more self-sufficient.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d00b24773c86" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/enhancing-developer-experience-and-machine-learning-workflows-with-mlops-d00b24773c86">Enhancing Developer Experience and Machine Learning Workflows with MLOps</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Your Data Team Needs MLOps]]></title>
            <link>https://medium.com/99p-labs/why-your-data-team-needs-mlops-f2a502d0fae0?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2a502d0fae0</guid>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Tue, 19 Dec 2023 14:14:27 GMT</pubDate>
            <atom:updated>2024-02-04T18:51:37.703Z</atom:updated>
            <content:encoded><![CDATA[<h4>And How to Create a MLOps Platform</h4><p>At 99P Labs one of our research areas is platform engineering and software development capability, DevOps, and MLOps. Every year we have iterated on versions of a platform that can support and enable all types of data scientists and software engineers. To follow along with our progress you can read our <a href="https://medium.com/99p-labs/kafka-and-protobuf-ingesting-data-into-our-platform-1ab8ea488bbe">blogs about platform engineering</a> and our <a href="https://medium.com/99p-labs/enabling-data-science-teams-of-any-size-43c8d8029b0">data science environment</a>.<br>Our preexisting data science environment shares a lot of similarities with a modern MLOps platform that can enable any Machine Learning engineer or data scientist the same. But after attending MLOps world in Austin, Texas (<a href="https://medium.com/p/0c671f3b53e3">Read the write up of the experience here</a>), I got to brainstorm concepts on how our team can expand our capabilities to help enable ML Engineers across the entire company become Machine Learning ready, to not worry about infrastructure or compute so they can focus on writing ML code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*p_wLAapMAbvYPwZmonEf1A.png" /><figcaption>MLOps: Code vs. Infrastructure</figcaption></figure><p>This picture is a great example describing the importance of ML Code as compared to how important the infrastructure is for deploying this code. A Machine Learning model can only be as good as the infrastructure surrounding it, supporting the development, experimentation, and “productionization” of the model.</p><p>As our team’s capability grows past data science and local Machine Learning development, we want to develop the golden standard of Machine Learning infrastructure and processes, to help create efficiency within our team and with our research partners. Right now time is wasted with trying to create a pipeline for every single experiment and running everything without automation. If every team we partner with and our in-house engineers all can speak the same standardized pipeline language, then projects can kick off more quickly, with much less confusion and effort required.</p><p>We want our Machine Learning engineers to focus on coding and models, not the supporting platform and infrastructure. This is the same concept with DevOps and platform engineering. To apply this concept to Machine Learning takes everything a step farther.</p><p>DevOps and MLOps both go a little something like this:</p><ul><li>Developer creates a script / application / code</li><li>Developer pushes code to git repository</li><li>Code automatically runs through a build pipeline</li><li>Code executes some process and has an output (artifacts) — This could be a model being created or an application compiled</li><li>Some final action — This could be the model being deployed or application running</li><li>Continuous iterations of this process</li></ul><p>Our existing platform is capable of this process in a DevOps sense, where we can automatically build and run applications, we can even run Spark and Python jobs this way. You could technically create models with this process but it’s not as clean and does not come with as many features.</p><p>There are toolkits that can extend a platform’s capability to become Machine Learning and MLOps ready. We decided to use Kubeflow but have looked at multiple projects incluing MLflow. Kubeflow is an open source container orchestration system that integrates directly with Kubernetes infrastructure. It deploys and manages tooling as containers, making it easier for users to access applications without having to know how the tooling runs. On the other hand, MLflow is a Python program that is agnostic of Kubernetes. While this simplicity provides benefits, it can also be a drawback for organizations that rely on Kubernetes infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*ibG3McUVkwYxuAYwKeDPbg.png" /><figcaption>Kubeflow v. MLflow</figcaption></figure><p>Kubeflow integrates directly into our Kubernetes based platform and everything is run as a container with complete scale and simple management. We also build all our applications and jobs as microservices and break everything into as small of components as we can. Starting from such a strong base with Kubernetes, containerization, and microservices around 5 years ago has made adoption of new processes and new applications very simple. That is one strong recommendation I would make to anyone starting MLOps: focus on creating a strong software platform first, which does not have to be complicated. Here are some of our recommendations:</p><ul><li>GitOps Mindset — Version everything</li><li>Git repositories (We use Gitea)</li><li>CI/CD Process (We use Argo CD and Jenkins)</li><li>Kubernetes platform (Scale, open source, easy management)</li><li>Infrastructure as code (We use Terraform)</li></ul><p>Since we had this infrastructure developed we were able to deploy Kubeflow on our platform and integrate it to our identity management pretty simply. Let me now discuss about what Kubeflow brings to our already mature platform:</p><ul><li>Automation of jobs</li><li>Its own pipeline framework, allows Python users to create automations of their scripts to be created simply and stored as infrastructure as code (Lets go KFP!!)</li><li>Collaborative environment, which lets Data Scientists and Machine Learning Engineers to work together on scripts before models are created. Very nice for experimentation.</li><li>User interface, allows noobs or experienced users to have control over their automation, scripts, models, deployments, etc.</li><li>Automatic resource allocation, the people writing the ML code do not need to worry about what resources they have available, they can just submit jobs and the Kubernetes cluster and MLOps (platform) engineers have already configured what resources they can use.</li><li>Model serving, it makes it very easy to go from model creation scripts and training to deploying a model into production (all from a UI, might I add)</li></ul><p>We hope in the next year to enable Data Scientists and push them into Machine Learning Engineers. Using the same concepts, we want to push Platform Engineers DevOps + MLOps role, where they enable both Machine Learning and software equally.</p><p>The line is becoming blurred for what a ML Engineer should do and what a Data Scientist should do, but modern platforms help enable the entire process and do not silo or box one into a certain role. On Kubeflow the experimentation environment allows both roles to work together to create scripts, modify and clean data, and create tests. The ML Engineer and MLOps Engineers can then work together to create infrastructure as code scripts to create pipelines for reuse and model deployment. None of these must be defined to a specific role, as Kubeflow allows you to control who has access to what process, and you could let a Data Scientist become more familiar with the process and give them access to deployments and pipelines, for example. MLOps really has just become allowing ML Engineers and DS pros to work on their models and code without having to worry about what infrastructure and resources they have access to.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2a502d0fae0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/why-your-data-team-needs-mlops-f2a502d0fae0">Why Your Data Team Needs MLOps</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MLOps World — Conference Summary]]></title>
            <link>https://medium.com/99p-labs/mlops-world-conference-summary-0c671f3b53e3?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/0c671f3b53e3</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Thu, 02 Nov 2023 18:32:18 GMT</pubDate>
            <atom:updated>2023-11-02T18:32:18.722Z</atom:updated>
            <content:encoded><![CDATA[<h3>MLOps World — Conference Summary</h3><p>Written By: Tony Fontana, Ryan Lingo &amp; Martin Arroyo</p><p>This blog is about my experience at MLOps World, an annual gathering of leading minds in Machine Learning Operations (MLOps). Hosted this year in Austin, TX, this conference provided a unique opportunity to learn, network, and bring back invaluable insights to my role at 99P Labs.</p><p>At 99P Labs we’re attempting to shape the future of mobility. Our work spans a range of areas, from human-computer interaction and green energy to software-defined mobility. As a Platform Engineer on the Software-Defined Mobility team, my year has been filled with diverse projects. We’ve been enhancing our data platform, expanding machine learning capabilities, and focusing on big data, sensor integration, and cloud computing. Our aim is to enable collaboration both within our team and with other teams working in similar research domains.</p><p>The linchpin that brings cohesion to these varied endeavors is MLOps — Machine Learning Operations. It’s become a focal point at 99P Labs, guiding us as we navigate the complexities of integrating machine learning into our operational workflows. This blog will delve into the key insights I gained from attending the MLOps World conference, and how these are influencing our approach to MLOps at 99P Labs.</p><h3>What is MLOps?</h3><p>After setting the stage with the broader goals of 99P Labs and the relevance of MLOps, you might be wondering: What exactly is MLOps? If you’re a Data Scientist, ML Engineer, or even just tech-curious, understanding MLOps can provide a new lens through which to view machine learning projects. Here are some foundational aspects that define MLOps:</p><p><strong>Automated Workflows</strong>: At the core of MLOps are automated workflows that streamline the model’s lifecycle from training to deployment. By minimizing manual intervention, these workflows significantly accelerate operational speed, allowing teams to focus on innovation rather than routine tasks.</p><p><strong>Model Versioning</strong>: In a field as dynamic as machine learning, keeping track of model iterations is crucial. Model versioning enables this by maintaining a detailed record of each model’s development, making it easier to roll back changes or understand performance variations.</p><p><strong>Model Monitoring</strong>: Once a model is in production, the work is far from over. Continuous monitoring ensures that the model performs as expected and adapts to new data patterns, thus maintaining its utility and effectiveness over time.</p><p><strong>Model Governance</strong>: Last but not least, MLOps also encompasses model governance, ensuring that all machine learning activities align with regulatory requirements and organizational policies. This is particularly vital in sectors where data privacy and security are paramount.</p><p>So, MLOps is not just a buzzword; it’s a multidimensional practice that intersects with various aspects of machine learning, from development to deployment and maintenance. As you’ll see, the learnings from MLOps World have a direct impact on how we approach these elements at 99P Labs.</p><p>For those interested in diving deeper into the intricacies of MLOps, I highly recommend this <a href="https://pages.ubuntu.com/rs/066-EOV-335/images/A_guide_to_MLOps_Canonical.pdf?_ga=2.25253821.1451693207.1677686496-763264821.1639074975">comprehensive guide to MLOps from Canonical.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*p9JEti2MSvz8biof" /></figure><h3>Key Learnings from MLOps World</h3><p>The MLOps World conference was an eye-opening experience that provided deep insights into the rapidly evolving field of MLOps. In this section, I’ve distilled my key takeaways into five overarching themes: The New Paradigm of Machine Learning, Understanding MLOps, Advanced Concepts, Practical Challenges, and Tools &amp; Technologies.</p><ol><li><strong>The New Paradigm of Machine Learning</strong></li></ol><p><strong><em>The Shift to Machine Learning 2.0 (GenAI)</em></strong>: The most talked-about subject this year was the shift from traditional Machine Learning to Machine Learning 2.0, commonly referred to as GenAI. GenAI is not just about using text and video for model training; it’s about providing context and continuously fine-tuning foundation models for deployment in production environments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bsdvfpEML6npXBn_" /><figcaption>Photo from UbiOps</figcaption></figure><p><strong>2. Understanding MLOps</strong></p><p><strong><em>Definition of MLOps</em></strong>: MLOps, an amalgamation of Machine Learning and Operations, is revolutionizing how we operationalize the entire Machine Learning lifecycle. It focuses on seamlessly integrating AI models into production environments, thereby enhancing efficiency, reliability, and scalability.</p><p><strong><em>Role of Machine Learning in Data Teams</em></strong>: The future of data teams is closely aligned with MLOps, emphasizing the transformative power of integrating Machine Learning into operational workflows. This fusion is driving a new standard for efficiency, reliability, and scalability in managing and deploying ML models.</p><p><strong>3. Advanced Concepts</strong></p><p><strong><em>Generative AI (GenAI) vs. ML Ops</em></strong>: GenAI marks the next evolutionary phase in artificial intelligence, featuring models with enhanced self-learning and adaptive capabilities that minimize the need for ongoing human intervention. In contrast, MLOps concentrates on the operational dimensions of Machine Learning. It aims to optimize the entire ML lifecycle, emphasizing the reproducibility, scalability, and reliability of ML models within production settings.</p><p><strong><em>Data-centric AI vs. Traditional ML</em></strong>: Data-centric AI is reshaping the focus towards utilizing high-quality, diverse, and relevant data for training AI models, highlighting the importance of data quality over intricate algorithms. In contrast, Traditional ML often leans more on the strength of powerful algorithms, sometimes at the expense of overlooking the critical role that high-quality data plays in model development.</p><p><strong>4. Practical Challenges</strong></p><p><strong><em>Drift Detection Problem</em></strong>:The drift detection issue involves the complex challenge of identifying discrepancies or deviations in data distributions over time, especially in ML models deployed in real-world settings. Recognizing such shifts is essential for maintaining both the accuracy and relevance of models as the underlying data evolves.</p><p><strong><em>Data Cleaning Challenges</em></strong>: In the realm of Machine Learning, data cleaning is a multifaceted task that involves preprocessing and refining raw data to ensure its quality, consistency, and relevance for effective model training. The primary challenges include handling missing or erroneous data, maintaining data integrity, and managing large volumes of data from diverse sources.</p><p><strong><em>Challenges Serving an ML Model</em></strong>: Serving an ML model comes with a multi-layered set of challenges. These include ensuring model accuracy in real-time scenarios, maintaining scalability, addressing resource constraints, navigating regulatory compliance, and adapting to continually changing data patterns.</p><p><strong>5. Tools &amp; Technologies</strong></p><p><strong><em>Kubeflow Charmed, Cleanlab/ML Server</em></strong>: Charmed Kubeflow is an open-source MLOps platform that converts various data science workflow steps into Kubernetes jobs. As an official distribution of the upstream Kubeflow project, it offers a simplified, portable, and scalable solution for ML deployments. Its capabilities span from experimentation using Notebooks to training through Kubeflow Pipelines and tuning. Cleanlab and ML Server complement this by serving as effective tools for deploying and managing ML models in production environments.</p><p><strong><em>Pipeline Observability</em></strong>: Pipeline observability entails the systematic monitoring, tracking, and analysis of the entire ML pipeline. This practice enables enhanced insights into workflow performance, helps in identifying bottlenecks, and assures both the reliability and efficiency of the overall process.</p><p><strong><em>Rules for a Platform</em></strong>: Establishing a robust platform necessitates adherence to key principles like scalability, reproducibility, security, flexibility, and interoperability. Beyond these fundamentals, a successful platform also needs to facilitate seamless collaboration, enable efficient resource management, and uphold transparent and auditable processes. These elements collectively foster innovation within a data-driven ecosystem.</p><p>In summary, MLOps World offered a multifaceted view into the future of Machine Learning Operations. From the paradigm shift towards GenAI to the practical challenges and tools shaping the field, the conference was a goldmine of insights. These learnings are not just theoretical but are directly influencing our projects and focus areas at 99P Labs. Speaking of influence, one of the standout moments of the conference was the enlightening talk by D. Sculley, which we’ll dive into next.</p><h3>D. Sculley’s Talk</h3><p>One of the highlights of MLOps World was a compelling talk by D. Sculley, the CEO of Kaggle and the person who coined the term ‘MLOps.’ His presentation offered a foundational understanding of what MLOps should entail. Contrary to some views, Sculley emphasized that MLOps is not just about building infrastructure but extends to practices, routines, and information gathering. The ultimate goal, he said, is to ‘automate toil and reduce human error.’</p><p>Sculley also shared several key takeaways that resonated deeply. He pointed out that MLOps is not merely about constructing process management tools. Even simple models, he noted, can have complex interactions with the real world. Establishing stable baselines for reference is not just essential but achievable with careful planning. Moreover, the challenges are only escalating in the era of GenAI and Large Language Models (LLM).</p><p>One of the most striking points was the importance of asking the right questions and making judicious judgments. Sculley stressed the need to explore multiple perspectives on the same issue, underscoring that this is a crucial and meaningful role for humans in the MLOps process. And, simplifying the essence of a good ML model, he stated that it should be ‘the least complex model that serves needs.’</p><p>Sculley’s talk was not just enlightening but also validated many of the trends and challenges we’ve been observing at 99P Labs.</p><h3>Conclusion</h3><p>As I wrap up this reflection on MLOps World, it’s evident that the conference was more than just a learning opportunity — it was a catalyst for change and innovation. From the paradigm-altering concept of GenAI to the nuanced insights from industry stalwarts like D. Sculley, the event offered a comprehensive look into the state and future of MLOps. These revelations are not confined to the realm of theory; they have immediate and far-reaching implications for our ongoing projects at 99P Labs.</p><p>The ever-evolving landscape of MLOps, especially in the context of GenAI and Large Language Models, presents both challenges and opportunities. As we continue to navigate this exciting field, the key takeaways from the conference will serve as guideposts, informing our strategies, methodologies, and decision-making processes.</p><p>Thank you for joining me on this journey through MLOps World. As we forge ahead, these learnings will undeniably shape our approach to software-defined mobility, driving us closer to a future where machine learning is seamlessly integrated into every facet of our operations.</p><p>We value your interest in 99P Labs and appreciate your time spent reading our blog. If you have any questions, concerns, or would like to discuss potential collaborations, we encourage you to reach out to us. You can connect with us on <a href="https://www.linkedin.com/company/99p-labs">LinkedIn</a> or <a href="https://medium.com/99p-labs">Medium</a> to stay updated on our latest research and innovations. Additionally, you can email us at research@99plabs.com to initiate a conversation. We are always excited to engage in meaningful discussions and explore exciting opportunities.</p><p>Thank you for your support, and we look forward to hearing from you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0c671f3b53e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/mlops-world-conference-summary-0c671f3b53e3">MLOps World — Conference Summary</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enabling Data Science Teams of Any Size]]></title>
            <link>https://medium.com/99p-labs/enabling-data-science-teams-of-any-size-43c8d8029b0?source=rss-a65e9be95d2c------2</link>
            <guid isPermaLink="false">https://medium.com/p/43c8d8029b0</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[apache-spark]]></category>
            <dc:creator><![CDATA[Tony Fontana]]></dc:creator>
            <pubDate>Thu, 06 Apr 2023 13:49:55 GMT</pubDate>
            <atom:updated>2023-04-06T13:49:55.122Z</atom:updated>
            <content:encoded><![CDATA[<p>As our data science team transitioned to open source and free tools, we realized that our AWS EMR was the final item we needed to let go of. Although it allowed for a fast startup of a spark cluster, it posed more challenges than benefits. Apart from its exorbitant cost, we faced numerous technological hurdles, including difficulties upgrading and keeping current versions of spark, the inability to use unsupported applications, and the complex setup process for new users.</p><p>The onboarding process for EMR was found to be cumbersome and slow. To access the EMR, users had to first acquire AWS credentials, and then use them to generate a key for accessing the SSH server, which was locked with only one entry for SSH. The process of configuring compute resources for Spark jobs was challenging, and importing Python packages that were not supported by AWS proved to be very difficult. Furthermore, the process of importing individual datasets to the EMR and running Spark commands from the terminal was time-consuming, precluding quick visualizations. Lack of versioning and Git support further complicated the process of creating and maintaining scripts, leading to unnecessary struggles with collaboration among team members. This entire process of onboarding a new user could take up to a week, and required significant effort to bring them up to speed.</p><p>We resolved to address these issues by determining our team’s needs and creating a platform that we could control, resulting in an onboarding process of 20 minutes or less. Our data science team primarily utilized Spark, Trino, and Python/Pyspark for their data analytics, and we opted to run these tools on our Kubernetes cluster. By shifting Trino to our platform a year earlier, we had already achieved remarkable improvements in speed and cost over EMR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*6zAki98xzm16Jbw-0xryFw.png" /><figcaption>New Spark Workflow</figcaption></figure><p>With the collaboration of our data science team, we created a diagram that illustrated our new workflow. To submit spark jobs, users could host their python environment and run Jupyterlab in a docker container on their local machine. We provided them with a preconfigured docker image that automatically connected to our platform and spark cluster using Sparkmagic and Lighter/Livy. This simplified the process for users, who only had to pull and run the docker image to spawn a Jupyterlab app on their machine, open it in their browser, and create a new Pyspark notebook that automatically connected to our spark cluster.</p><p>Our new workflow allows data science teams to configure their spark jobs and compute resources with options to request memory and CPU from the cluster and customize spark settings. Sparkmagic enables users to create visualizations computed on the spark cluster, which was not supported by AWS EMR.</p><p>The benefits of this new workflow include:</p><ul><li>Streamlined setup process without the need for configuration or permissions.</li><li>Simplified scripting through a visual notebook and terminal access.</li><li>Version control of scripts, increased ease of collaboration via git integration</li><li>Access to a comprehensive catalog of example code from previous projects.</li><li>Immediate access to datasets via our Spark cluster.</li><li>Rapid visualization through Pyspark, facilitating quick insights.</li><li>Quick and easy setup that takes only 20 minutes.</li><li>Unlimited scaling potential for future growth.</li><li>Compatibility with any Python package.</li></ul><p>Our deployment of this stack on our platform is completely customizable and scalable and costs only a fraction of what we paid for EMR. Built with open source tools, we are excited to see how our data science team will utilize this new workflow. For more information, please contact us at <a href="/service/mailto:support@99plabs.com">support@99plabs.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43c8d8029b0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/99p-labs/enabling-data-science-teams-of-any-size-43c8d8029b0">Enabling Data Science Teams of Any Size</a> was originally published in <a href="https://medium.com/99p-labs">99P Labs</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>