<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yaniv Vaknin on Medium]]></title>
        <description><![CDATA[Stories by Yaniv Vaknin on Medium]]></description>
        <link>https://medium.com/@yaniv.vaknin?source=rss-3ad8135b9742------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pWvJ8ePM1B0-E6oqlzTW0w.jpeg</url>
            <title>Stories by Yaniv Vaknin on Medium</title>
            <link>https://medium.com/@yaniv.vaknin?source=rss-3ad8135b9742------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:36:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yaniv.vaknin/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Unravel LLM building blocks]]></title>
            <link>https://medium.com/@yaniv.vaknin/unravel-llm-building-blocks-8da807e9e574?source=rss-3ad8135b9742------2</link>
            <guid isPermaLink="false">https://medium.com/p/8da807e9e574</guid>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Yaniv Vaknin]]></dc:creator>
            <pubDate>Tue, 22 Aug 2023 20:36:04 GMT</pubDate>
            <atom:updated>2023-08-22T20:36:04.758Z</atom:updated>
            <content:encoded><![CDATA[<p>The total amount of data created, captured, copied, and consumed globally is forecast to increase rapidly, <a href="https://www.statista.com/statistics/871513/worldwide-data-created/">reaching 120 Zettabytes in 2023</a>. By 2025, global data creation is expected to reach 180 Zettabytes..</p><p>According to estimates, <a href="https://www.mongodb.com/unstructured-data">80 to 90 percent of data is unstructured</a> ( e.g. text, image, etc.), meaning it is unorganized and inaccessible to conventional data analytics tools.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zhURgrTexJVENJBmLtM0jQ.jpeg" /><figcaption>Created by DALL-E</figcaption></figure><p>LLM (Large Language Model) leverages a unified way to consume, analyze, search and train this volume of data, LLMs use a language for a unified representation of unstructured data — Embeddings.</p><p>This topic has been the subject of many papers. In this article, my aim is to provide you with the essential foundations and practical code examples that will simplify the building blocks of LLM. This will ensure you have the fundamental knowledge and smooth route to navigate a technical discussion like “On how many tokens GPT-4 was trained on?” or “What is Open AI’s ADA (hint: embedding ) model ?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cZom-h_iE9iyke9g9AYV3Q.png" /><figcaption>Image by author</figcaption></figure><h3>What is an embedding?</h3><p>Google’s team set the stage with <a href="https://jalammar.github.io/illustrated-word2vec/">Word2vec</a> and the foundational paper “A<a href="https://arxiv.org/abs/1706.03762">ttention is all you need</a>”. These papers build the pillars for the emerging language <strong>“Embeddings”</strong>.</p><p>Word and sentence embeddings are the building blocks of language models. Embeddings map items of unstructured data to high-dimensional real vectors.</p><p>Word embeddings are vector representations of words, where each dimension represents a different feature or aspect of the word’s meaning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QrWRbYko0naaZtc5hGYOkw.png" /><figcaption>Word embedding — 50 features (<a href="https://jalammar.github.io/illustrated-word2vec/">Source</a>)</figcaption></figure><p>Embeddings are typically created using neural network models that learn to map words or <strong>tokens</strong> to a high-dimensional vector space.</p><h3>Embeddings Vs. Tokens</h3><p>OK, that’s great but how can we decide how to split the input data? It can be easy for frequent and short words, but for non-frequent words we need a split mechanism.</p><p>This is where tokenization comes into play where each word is tokenized into individual tokens and then embedded into a vector representation.</p><p>Tokens are common sequences of characters found in text. One token is ~4 characters, or one token equals 0.75 words by mean.</p><p>To calculate how many tokens are included in your text you can use <a href="https://github.com/openai/tiktoken">Tiktoken</a>, a fast <a href="https://en.wikipedia.org/wiki/Byte_pair_encoding">BPE</a> (Byte Pair Encoding) tokenizer for use with OpenAI models.</p><p>The open source version of Tiktoken can be installed from PyPI:</p><pre>pip install tiktoken</pre><p>This function returns the number of tokens in a text string:</p><pre>import tiktoken<br>def num_tokens_from_string(string: str, encoding_name: str) -&gt; int:<br>    encoding = tiktoken.get_encoding(encoding_name)<br>    num_tokens = len(encoding.encode(string))<br>    return num_tokens</pre><p>If we want to know how many tokens are in the sentence “tiktoken is awesome”, we can call the function above in the following way, in this case we used cl100k_base.</p><pre>print(num_tokens_from_string(&quot;tiktoken is awesome!&quot;, &quot;cl100k_base&quot;))</pre><pre>6</pre><p>Popular OpenAI models likegpt-4, gpt-3.5-turbo, text-embedding-ada-002 use cl100k_base encoding model.</p><p>You can use this <a href="https://platform.openai.com/tokenizer">visual tool</a> provided by OpenAI. For example: the sentence “Mary Poppins Supercalifragilisticexpialidocious”, comprises 15 tokens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oJTFGyvbLfdnCuDzNl-mqA.png" /></figure><p>Now that we understand how to represent a word as a vector of numbers — a word embedding — and have a mechanism to split it into tokens, this is where “<strong>Sentence Embeddings” </strong>come into play. A sentence embedding is just like a word embedding, except it associates every sentence with a vector full of numbers. We encode the position of that word in the input sentence into a vector, and add it to the word vector. This way, the same word at a different position in a sentence is encoded differently.</p><p>The following code generates a vector of 384 features for each of the sentences [“I like dogs”, “I like cats”]. It is imperative to know that there are different models with various numbers of features (~96–4096 features).</p><p>Consider the following tradeoffs when choosing your model:</p><p>Many features -&gt; higher recall (accuracy) -&gt; more memory (higher latency / slower performance)</p><pre>pip install -U sentence-transformers</pre><pre>from sentence_transformers import SentenceTransformer<br>sentences = [&quot;I like dogs&quot;, &quot; I like cats&quot;]</pre><pre>model = SentenceTransformer(&#39;sentence-transformers/all-MiniLM-L6-v2&#39;)<br>embeddings = model.encode(sentences)<br>print(embeddings)</pre><p>This is the short output:</p><pre>[[-5.84880412e-02 -2.79528778e-02  6.88477978e-02  2.85013970e-02<br>  -6.79694414e-02 -2.25785817e-03  7.21920580e-02 -5.46673220e-03<br>   1.04574956e-01  5.83925620e-02  7.27049336e-02 -6.51337430e-02</pre><h3>Summary</h3><p>This article discusses the increasing amount of data generated globally and the need for a unified way to consume and train unstructured data and LLMs. It emphasizes the importance of embeddings, which are vector representations of unstructured data, such as words or sentences. Tokenization is introduced as a mechanism to split input data, especially non-frequent words. Tokens are sequences of characters found in text, and they are embedded into vector representations. Tiktoken is a tokenizer for OpenAI models, and provides an example code snippet to count the number of tokens in a text string. Additionally, it introduces the concept of sentence embeddings, which associate every sentence with a vector of numbers. We provide code examples for generating sentence embeddings using the SentenceTransformer library.</p><p>If someone will ask you :-)</p><ul><li>According to this <a href="https://www.semianalysis.com/p/gpt-4-architecture-infrastructure">article</a> OpenAI trained GPT-4 on ~13 Trillion tokens</li><li>Text-embedding-ada-002 is a new embedding model from OpenAI that replaces separate models for text search, text similarity, and code search</li></ul><p>References:</p><ul><li><a href="http://vectors.nlpl.eu/explore/embeddings/en/#">WebVectors: distributional semantic models online</a></li><li><a href="https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2">sentence-transformers/all-MiniLM-L6-v2 · Hugging Face</a></li><li><a href="https://projector.tensorflow.org/">Embedding projector - visualization of high-dimensional data</a></li></ul><h4><a href="https://mlearning.substack.com/about">WRITER at MLearning.ai</a> / <a href="https://open.substack.com/pub/mlearning/p/i-tested-800-plugins-chatgpt-the-best-openai?r=z7zu8&amp;utm_campaign=post&amp;utm_medium=web">800+ AI plugins</a> /<a href="https://open.substack.com/pub/mlearning/p/how-ai-artist-art-copyright-midjourney-protect?r=z7zu8&amp;utm_campaign=post&amp;utm_medium=web"> AI art Copyright</a></h4><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8da807e9e574" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is Machine Learning the Silver Bullet for Cyber Attacks?]]></title>
            <link>https://medium.com/@yaniv.vaknin/is-machine-learning-the-silver-bullet-for-cyber-attacks-a400227a7143?source=rss-3ad8135b9742------2</link>
            <guid isPermaLink="false">https://medium.com/p/a400227a7143</guid>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Yaniv Vaknin]]></dc:creator>
            <pubDate>Sun, 13 Nov 2022 18:36:10 GMT</pubDate>
            <atom:updated>2022-11-14T10:53:33.723Z</atom:updated>
            <content:encoded><![CDATA[<p>I’m sorry to disappoint you. Unfortunately, Machine Learning (ML) isn’t the silver bullet for cybersecurity…</p><p>Nevertheless, ML is increasingly being adopted to combat evolving cyber threats. Why is ML becoming an essential component of security products?</p><p>In the next few sections, I review the challenges and opportunities, and discuss why now more than ever is the ideal time to build ML based cybersecurity solutions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bDvDRgpDOyLNhKVjxRO2bg.png" /><figcaption>Silver Bullet by DALL-E</figcaption></figure><p><strong>It’s still day one for ML and Cyber</strong></p><p>According to <a href="https://www.gartner.com/smarterwithgartner/gartner-top-10-trends-in-data-and-analytics-for-2020?utm_medium=social&amp;utm_source=twitter&amp;utm_campaign=SM_GB_YOY_GTR_SOC_SF1_SM-SWG&amp;utm_content=&amp;sf240089296=1">Gartner</a>, by the end of 2024, 75% of enterprises will gain a competitive edge by shifting from piloting to operationalizing ML.</p><p>Similarly, companies that embrace ML for cybersecurity will gain a significant advantage. Many ML capabilities are particularly well suited for addressing cybersecurity challenges, including modeling user behavior (anomaly detection), email phishing (classification), threat scoring (linear regression), etc.</p><h3>Challenges</h3><p>Historically, security has been a binary rule-based system in which the state is either 0 (this file is benign) or 1 (congratulations, you have a virus). Complex systems define these classes based on a set of rules.</p><p>But, how can you face this challenge at scale with more than 450,000 <a href="https://www.av-test.org/en/statistics/malware/">new malware types identified per day</a> and more than 1.3 billion malware types already out there? How can Security Operations Center(SOC) teams handle the explosion in new types and the sheer scale of attacks?</p><p>Furthermore, security experts need to understand why a file or an event is classified as malware or as an anomaly.</p><p>Machine Learning is designed to enable machines to learn from a set of pre-observed classes. But, what happens if a type of data wasn’t present in the pre-observed classes? Even the most sophisticated model is unable to predict new patterns, for example, a model trained to classify dogs will not be able to predict a zebra. This is precisely what zero-day attacks take advantage of.</p><p>The problem gets worse and worse. As security products evolve, attackers are not sitting on their hands, instead, they develop AI based attack vectors. For example, attackers use AI technologies to create malware capable of mimicking trusted system components.</p><blockquote>“We’re not being attacked by human beings anymore. Computers are attacking us, software is attacking us. The only way forward is using artificial intelligence” (CTO of AXA IT North Europe, Yorck Reuber)</blockquote><p>This quote is particularly interesting, due to the fact that it came from the CTO of AXA, one of the largest insurance carriers in Europe, which recently <a href="https://apnews.com/article/europe-france-technology-business-caabb132033ef2aaee9f58902f3e8fba">announced</a> that it would stop covering ransom payments under its cyber insurance policies.</p><p>The changing nature of ransomware attack vectors is illustrated in the graph below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/656/1*IYIAc8MI7KJXsAHOr9xLbw.png" /><figcaption>Source: <a href="https://www.coveware.com/blog/ransomware-attack-vectors-shift-as-new-software-vulnerability-exploits-abound">https://www.coveware.com/blog/ransomware-attack-vectors-shift-as-new-software-vulnerability-exploits-abound</a></figcaption></figure><p>The leading attack vectors are Remote Desktop Protocol (RDP) compromise, which declined from 80% to 40%, email phishing, which soared to more than 50% during 2020, and software vulnerability (It’s no secret that 90% of generated code is based on reusable code from open source, GitHub repositories, etc.).</p><h3>Opportunities</h3><p>I must emphasize that ML alone is rarely a viable solution. Other measures, including human intervention will likely be necessary. For example, in some use cases, it’s a good practice to have a human evaluate the machine predictions which fell below a certain threshold.</p><p>Companies that build a coherent ML strategy and overcome the many challenges of using ML in production at scale (you can read about the challenges of MLOps <a href="https://medium.com/mlearning-ai/why-is-mlops-a-strategic-goal-for-tech-leaders-c2cf3262defa">here</a>) will be able to offer accurate alerts, with a low rate of false positives, and will be able to operate with smaller SOC teams, thereby lowering manpower costs.</p><p>Another area with growth potential is Zero-Shot Learning (ZSL), one of ML’s most fascinating domains. Simply put, ZSL allows a model to recognize what it hasn’t seen before.</p><p>ZSL could be an effective solution for zero-day vulnerability. ZSL could also help handle the alarm flooding problem faced by cybersecurity systems like Security Information Event Management (SIEM) and Intrusion Detection (IDS).</p><p>This <a href="https://arxiv.org/abs/2106.14647">article</a> suggests a new approach to automatically detect alarm labels generated in SIEM and associate them with specific attack types. Using this approach, we can identify attacks, determine what features contribute to classification, and categorize them — without any prior knowledge.</p><p>ZSL proposes a zero-shot approach for categorizing novel attacks into specific new classes based on feature influence. Using the resulting system, attacks can be distinguished from normal traffic and labels can be generated automatically based on features that contribute to an attack.</p><h3>Why is now the right time?</h3><p>Now, more than ever, we are living at a time where developers have fertile ground to build ML based cyber solutions. This is due to three main factors.</p><p>The first factor is AI ML companies democratizing ML and enabling ML practitioners to innovate faster. ML platforms like SageMaker expedite the interval between research to PoC and PoC to production. AutoML enables developers to run quick experiments with fast pass/fail results.</p><p>The second factor is the vibrant ecosystem. NVIDIA recently launched NVIDIA <a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus">Morpheus</a>, a bespoke ML framework for cyber use cases that enables cybersecurity developers to create optimized AI pipelines for filtering, processing and classifying large volumes of real-time data and a pre-trained model.</p><p>The third factor is the rise of Natural Language Processing (NLP). One of the biggest challenges in the cyber domain is the lack of labeled data. However, the practice of Self-Supervised Learning (SSL) is growing. In essence, SSL is a machine learning process where the model trains itself to learn one part of the input from another part of the input. Today you can find pre-trained BERT models for security domain like SecBERT and <a href="https://huggingface.co/jackaduma/SecRoBERTa?text=Paris+is+the+%3Cmask%3E+of+France">SecRoBERTa</a> on the Hugging Face website.</p><p>You can read more here:</p><p><a href="https://towardsdatascience.com/understanding-zero-shot-learning-making-ml-more-human-4653ac35ccab">Understanding Zero-Shot Learning — Making ML More Human</a></p><p><a href="https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus">https://catalog.ngc.nvidia.com/orgs/nvidia/teams/morpheus/containers/morpheus</a></p><ul><li><a href="https://huggingface.co/jackaduma/SecRoBERTa?text=Paris+is+the+%3Cmask%3E+of+France">jackaduma/SecRoBERTa · Hugging Face</a></li><li><a href="https://www.coveware.com/blog/ransomware-attack-vectors-shift-as-new-software-vulnerability-exploits-abound">Ransomware Attack Vectors shift as New Software Vulnerability Exploits Abound</a></li></ul><p><a href="https://medium.com/mlearning-ai/why-is-mlops-a-strategic-goal-for-tech-leaders-c2cf3262defa">https://medium.com/mlearning-ai/why-is-mlops-a-strategic-goal-for-tech-leaders-c2cf3262defa</a></p><ul><li><a href="https://arxiv.org/abs/2106.14647">Zero-shot learning approach to adaptive Cybersecurity using Explainable AI</a></li><li><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a400227a7143" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why is MLOps a strategic goal for tech leaders?]]></title>
            <link>https://medium.com/@yaniv.vaknin/why-is-mlops-a-strategic-goal-for-tech-leaders-c2cf3262defa?source=rss-3ad8135b9742------2</link>
            <guid isPermaLink="false">https://medium.com/p/c2cf3262defa</guid>
            <category><![CDATA[ml-so-good]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Yaniv Vaknin]]></dc:creator>
            <pubDate>Tue, 22 Feb 2022 18:35:18 GMT</pubDate>
            <atom:updated>2023-08-24T06:08:20.281Z</atom:updated>
            <content:encoded><![CDATA[<p>Congratulations! You’ve finally succeeded in building a Machine Learning model. Your metrics look good, and you’re ready to launch a revolutionary ML-based product to create a competitive advantage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/390/1*mstrF_xXI2BJ4X1vvJvDwQ.png" /><figcaption>Data science team after they developed the “secret sauce” (<em>Image by author)</em></figcaption></figure><p>Well, guess again.</p><p>MLOps is the elephant in the room, where companies can fail in launching an ML based solution. According to <a href="https://www.gartner.com/en/newsroom/press-releases/2020-10-19-gartner-identifies-the-top-strategic-technology-trends-for-2021#:~:text=Gartner%20research%20shows%20only%2053,a%20production%2Dgrade%20AI%20pipeline.">Gartner</a> (October 2020), only 53% of projects make it from prototypes to production, and that’s at organizations with some level of ML experience. so if you are a tech manager: CEO, VP R&amp;D, Director of engineering, and you would like to be part of this statistics, you can stop here.</p><p>MLOps can refer to Machine Learning OPerationS, but this definition can be tricky, since it encapsulates a substantial set of practices, skills, roles, and employees to fulfill them.</p><p>In a recent study, <a href="https://www.rtinsights.com/key-requirement-mlops/">Forrester</a> found that 73% of respondents said MLOps adoption would keep them competitive, while 24% said it would make them an industry leader.</p><p>As there are over <a href="https://huyenchip.com/2020/12/30/mlops-v2.html">284 ML tools</a> (December 2020), this article will not cover the tools, but rather the pillars of challenges, concepts, and best practices that can be used to help you build a sustainable ML strategy.</p><p>In the next paragraphs, I will share a bespoke approach for building solid ML <strong>O</strong>btainable <strong>P</strong>roductionable <strong>S</strong>calable strategy for tech leaders. The rationale behind this model is building a ML strategy in a gradual way, based on 3 stages:</p><p>1. Obtainable — 1 ML model / Beta end users (1:few)</p><p>2. Productionable — 1 ML model / many end users (1:many)</p><p>3. Scalable — many ML models / many end users (many:many)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*o5ZrZc6IW13W09KnJ7xJUw.png" /><figcaption>MLOps model (<em>Image by author)</em></figcaption></figure><p><strong>Obtainable</strong></p><p>Data science teams aim to develop the most accurate state of the art models. They are using various techniques to push the limits, such as hyperparameter optimization. But, In order to embed the ML model into the company’s product, stakeholders should orchestrate the relationship between the ML model and the product.</p><p>The main challenge is to build a layer of business logic in a way that the ML model will be an inherent component which empowers the end user to “<strong>obtain”</strong> a better experience with the ML based product, while keeping the <a href="https://www.profitwell.com/recur/all/unit-economics">unit economics</a> worthwhile.</p><p>Then the engineering team should translate the business logic to an architecture that will comply with the company’s data governance, compute infrastructure such as Kubernetes, API integrations, gradual implementation, human in the loop and more.</p><p>“Rillow” is a real estate company, a team of data scientists developing a model to predict house pricing, in order to find undervalued assets, potentially buying and selling the assets (Any similarity to Zillow’s <a href="https://www.geekwire.com/2022/commentary-how-homeowners-defeated-zillows-ai-ultimately-leading-to-zillow-offers-demise/">story</a> is purely coincidental). During the PoC the F1 score was 95%, but during the implementation the challenge was how to infuse the ML model into the product. Taking a decision to buy real estate at an overvalued price can be a painful move. In order to avoid a failure, a mechanism based on gradual adoption is needed.</p><blockquote>Delusional perception of ML as a plug-and-play technology with immediate returns, can lead to bitter disappointment</blockquote><p>A viable deployment process can consist of three phases: shadow, canary and full deployment.</p><p>Shadow deployment is a process in which we send production traffic to the newly tested model. This way, data scientists can get real feedback, handle outliers and calibrate the model.</p><p>Canary deployment is often a synonym for A/B testing. During this stage, we expose the new model to a small percentage of end users. We would like to see improvement in the predefined KPIs, only then can we increase the amount of traffic handled by the new model.</p><p>Full deployment is the stage where we are handling 100% traffic. It’s a best practice to keep the old version so we can easily roll back to the previous model.</p><p>A gradual rollout of the first ML model might seem tedious, but in that case the longest path is the shortest path.</p><p>An easy way to demystify the ML <strong>O</strong>btainable <strong>P</strong>roductionable <strong>S</strong>calable concept is with an example of building a manufacturing line for cakes.</p><p>We all love cakes.</p><p>First and foremost, you must have a secret sauce (ML model). The next challenge is to infuse it in the cake and get feedback from the customers. Only then can you build the first manufacturing line. It will be worthless to productionize it before you have feedback from customers. The next phase is to scale the multi “secret sauces” to multi manufacturing lines with millions of customers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*bgVdpF6b9bce305irhl_Bw.png" /><figcaption>Obtainable — integrate the ML model into the product and expose it to end users (<em>Image by author)</em></figcaption></figure><p><strong>MLOps tools for Obtainable models</strong></p><ol><li><strong>Experiments management</strong> — organize, visualize, compare and evaluate machine learning experiments</li><li><strong>Model registry</strong> — central repository that allows developers to manage model versions and publish various production-ready models</li></ol><p>* It goes without saying that some of the tools relevant for all stages</p><p><strong>Key takeaway</strong></p><blockquote>“Is there a mechanism to ensure frictionless implementation of the ML model both for technical teams (gradual deployment) and business teams (business logic)?”</blockquote><p><strong>Productionable</strong></p><p>Once the model is ready for production, usually, the data science team hands off the model to the engineering team. This is a critical part, now the model will need to evolve and respond to the fast pace of data changes.</p><p>ML CI/CD is an aspect of machine learning operations that automatically and continuously preprocess data, train, tune, evaluate, approve, deploy and <strong>“productionize”</strong> machine learning models to adapt to changes in the data. The trigger for a re-build can be data change, model change, or code change (an essential component is integration with Git repositories). The predominant challenge of this stage is to remove the hurdle between research and production.</p><p>“Good cyber” is a cyber company with an ML based model to detect malwares or vulnerabilities. With more than <a href="https://dataprot.net/statistics/malware-statistics/">500,000 new malwares</a> per day, or major vulnerabilities like <a href="https://logging.apache.org/log4j/2.x/security.html">Log4J</a>, yesterday’s model will be ignorant of the rapid changes. Automation of the ML CI/CD is needed (the term continuous training (CT) for ML CI/CD might be a narrow definition of the concept, because it includes only the training phase ), Good Cyber’s example emphasizes the need for automation and an agile environment.</p><p>There are two types of ML CI/CD: event driven and time driven. Event driven is based on a reaction to an event(similar to <a href="https://en.wikipedia.org/wiki/Event-driven_architecture">event driven architecture</a>). The retraining process can be triggered by an accuracy degradation. In this case, the data scientist can analyze the root cause and decide to retrain the model with the new data.</p><p>Time driven is an automatic process in which the model pipeline is triggered every day/week/etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/424/1*LsuVym4CZANLuZCJx9gklw.png" /><figcaption>Productionable — serving multi end users (<em>Image by author)</em></figcaption></figure><p><strong>MLOps tools for Productionable models</strong></p><ol><li><strong>ML workflow orchestration</strong> — automate and manage workflows and pipeline infrastructure, in a collaborative interface.</li><li><strong>Model and data monitoring</strong> — a series of techniques that are deployed to better measure key model and data performance metrics and understand when issues arise.</li></ol><p><strong>Key takeaway</strong></p><blockquote>“Do you have a mechanism, which gradually moves the processes from manual to semi-automation/full automation??</blockquote><p><strong>Scalable</strong></p><p>Now that we are all set, it’s time to “<strong>scale”</strong>: more builders, more models, more end users.</p><p>It can be new models for new use cases or breaking the one monolith model into micro models such as model per customer (B2B) or cluster of customers (B2C).</p><p>Scale challenge can be solved by automation tools like auto scaling groups, optimizing the instances utilization with inference recommendation tools, templatizing infrastructure (e.g. CloudFormation, Terraform), sharing data, features and notebooks.</p><p>Sharing features is a practical way to share data and scale ML models, feature engineering is an iterative, time-consuming, and resource-intensive process. It requires both technical and domain knowledge.</p><p>It might be affordable when it’s a small team of data scientists and data engineers, but, when companies scale, the inefficiency of different teams working on the same features, reinventing features, can result in feature inconsistency, difference between feature definition in R&amp;D, Prod, and more.</p><p>Feature store is a data management layer for machine learning (Uber introduced Michelangelo in 2017 — the first feature store), enabling data engineers and data scientists to register, discover, use, and share features across a company. A feature store makes sure features are always up to date for predictions and maintains the history of each feature’s values in a consistent manner.</p><p>Sharing notebooks and experiments across teams becomes another critical parameter in the development velocity. Cross-organization collaboration is a key component to scale fast from one model to 10s/100s/1000s of models (nice <a href="https://aws.amazon.com/blogs/machine-learning/how-imperva-expedites-ml-development-and-collaboration-via-amazon-sagemaker-notebooks/">blog</a> about sharing notebooks).</p><p>Democratizing ML models with AutoML no code tools, can expand the reach of ML to <a href="https://blogs.gartner.com/carlie-idoine/2018/05/13/citizen-data-scientists-and-why-they-matter/">“Citizen data scientists”</a> (e.g. data analysts), though it’s not a classic MLOps tool, the added value of this tool is to offload “easy” workloads from the data science teams, and boost the ML adoption pace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*XuU2U5t7mlLbNh1UFWSAfg.png" /><figcaption>Scalable — serving multi end users with multi models (<em>Image by author)</em></figcaption></figure><p><strong>MLOps tools for Scalable models</strong></p><p><strong>1. Feature store</strong> — single pane of glass where you can manage all your features.</p><p><strong>2. AutoML</strong> — automates the selection, composition and parameterization of machine learning models.</p><p><strong>Key takeaway</strong></p><blockquote>“Do you have supportive infrastructure to scale ML models in an efficient and broad manner?”</blockquote><p><strong>Summary</strong></p><p>While it’s relatively no brainer to run a single ML model, if we multiply it by X experiments of hyperparameter optimization, multiply it by Y new datasets, multiply it by Z data scientists, and multiply it by W models, you get the picture, things get complicated.</p><p>To face this complexity, tech leaders must invest in ML tools and most importantly empower their team with the right knowledge to automate MLOps challenges.</p><p><a href="https://medium.com/mlearning-ai/mlearning-ai-submission-suggestions-b51e2b130bfb">Mlearning.ai Submission Suggestions</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c2cf3262defa" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>