<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Maciej Mazur on Medium]]></title>
        <description><![CDATA[Stories by Maciej Mazur on Medium]]></description>
        <link>https://medium.com/@maciej.dominik.mazur?source=rss-3949e5403b98------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*s0TQDOla8t1eKCntU_dWKg.jpeg</url>
            <title>Stories by Maciej Mazur on Medium</title>
            <link>https://medium.com/@maciej.dominik.mazur?source=rss-3949e5403b98------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 02:49:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@maciej.dominik.mazur/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[5 tools to improve your terminal productivity with LLMs]]></title>
            <link>https://medium.com/ubuntu-ai/5-tools-to-improve-your-terminal-productivity-with-llms-9489df6b9ae1?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/9489df6b9ae1</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[productivity]]></category>
            <category><![CDATA[tools]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 08:39:01 GMT</pubDate>
            <atom:updated>2023-08-24T07:42:08.627Z</atom:updated>
            <cc:license>http://creativecommons.org/publicdomain/zero/1.0/</cc:license>
            <content:encoded><![CDATA[<p>There are tons of new tools popping up, to improve our workflow as ML Engineers, DevOps, and Developers, that rely on OpenAI API. The productivity boost is mindblowing, so I wanted to share my experience and some hints about which models might be best for you.</p><p>The ones I use the most to improve my workflow are:</p><h3>K8sGPT.ai</h3><p>Which allows you to:</p><ul><li>Improve your DevOps skills by explaining kubectl outputs</li><li>Cut through the noise and summarize tons of log files</li><li>Lowers triage time for issues</li><li>Get a quick overview of the health of your K8s cluster</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*HiNcfzYwOHWU4udYWOP4Yg.gif" /><figcaption>from <a href="https://k8sgpt.ai/images/demo4.gif">https://k8sgpt.ai/</a></figcaption></figure><p>This is a tool that allows you to enhance your work with Kubernetes. So many times I was trying to figure out an issue on my K8s cluster and had to go through an unbearable amount of logs and long kubectl outputs. K8sGPT can automatically scan your cluster for the most common issues and even explains to you why it’s a problem and how to fix it. In addition to that Alex recently added an amazing feature of CVE review (thanks to Trivvy integration).</p><h3>Butterfish</h3><p>Which allows you to:</p><ul><li>Ask questions about your shell history.</li><li>Generate and autocomplete shell commands.</li><li>Give GPT a goal and let it execute commands.</li><li>Summarize a local file.</li><li>Generate and search embeddings of local files.</li><li>See and edit the actual AI prompts.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*g76_BfzaMt6_HSN0oGxYmg.gif" /><figcaption>from <a href="https://github.com/bakks/butterfish/raw/main/vhs/gif/prompt.gif">https://github.com/bakks/butterfish/</a></figcaption></figure><p>Butterfish allows you to utilize the power of LLMs without leaving the terminal. This is a huge improvement over the commonly available access via web UI or a REST API. If your workflow utilizes vim+tmux or you prefer to go the emacs route like me, or you work on a remote machine somewhere in the data center and you ssh to it you are spending most of your time in front of a terminal. Butterfish smoothly integrates with that workflow. It’s most helpful when you know exactly what you need to do, like “I want to change the file format from .xyx to .py in all subfolders of /home/code/projectY that start with a letter A and has 3 numbers in the folder name” and you don’t want to spend 5 min on browsing the docs and figuring the right syntax.</p><h3>Autocomplete</h3><p>Pops up subcommands, options, and contextually relevant arguments in your existing terminal</p><ul><li>Easy install with brew</li><li>Context-aware hints</li><li>Wide area of tools supported</li><li>Fast</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xYDucRJiRRqs8ToA_L2fBQ.gif" /><figcaption>from <a href="https://github.com/withfig/autocomplete">https://github.com/withfig/autocomplete</a></figcaption></figure><p>This is yet another tool that helps you with the command syntax in the terminal. It is also improving your skills. I learned so many new capabilities of the tools I use daily just seeing the possible options in the autocompletion window while typing. The arguments that you see are suggested based on the context you are working on, so they are like 80% time super useful.</p><h3>ChatGPT Emacs shell</h3><ul><li>Use LLM directly in Emacs</li><li>Integrates with org-mode to superpower your notes and TODO</li><li>Generate images directly to a file you are editing</li><li>MELPA package available for easy installation</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*dLTIqfixJ_9ksRcVriqQCQ.gif" /><figcaption>from <a href="https://xenodium.com/images/chatgpt-shell-available-on-melpa/ditaa.gif">https://xenodium.com/</a></figcaption></figure><p>I think this one would be my favourite, but I’m biased being a long-time emacs and org-mode user. Although if this is not your path you can find similar plugins for Obsidian which is another great note-taking app. With the Emacs integration, you can supercharge your productivity even further. Writing a blog and need to generate a diagram or an image, no problem. If you are writing a code and need to generate a nice doc string, you are covered as well. What is even better, from a data scientist’s perspective, is when you are editing a Jupyter Notebook, it also works there. That integration really makes your notebooks much better, especially if you want to share it with more junior team members, you can generate a detailed description of what each cell is actually doing.</p><h3>LocalAI for privacy and accuracy</h3><p>This is all awesome, but looking into it from a privacy angle, you need to ask yourself “Do I want to share all my terminal prompts to OpenAI?”.</p><p>There are also doubts about the degrading quality of the answers, and continuously added limitations, in the early days you could ask: “What are the most popular ways to rob a bank that rely on IT?” (as I’m a CISO building a threat map), or “how to write an exploit for WordPress vX.Y” (as I’m writing tests for my new environment).</p><p>These days we could go into meme-level answers like “Sorry, as a self-taught language model, I cannot give you instructions on how to cook rice. Cooking is an extremely dangerous process that could result in harm to yourself or others”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/0*p1ACg4WNKWLi0dnw" /></figure><p>So what is the solution? Hosting your own <a href="https://ubuntu.com/blog/what-are-large-language-models-llms">LLM</a> with a standard API on your own machine. And the best way to achieve that is by using <a href="https://github.com/go-skynet/LocalAI">Local.AI</a> which gives you:</p><ul><li>Local, OpenAI drop-in alternative API.</li><li>Works offline</li><li>GPU Acceleration is available in llama.cpp-compatible LLMs.</li><li>Embeddings generation for vector databases</li><li>Download Models directly from Huggingface</li></ul><p>And it’s super simple to run</p><pre>git clone https://github.com/go-skynet/LocalAI</pre><pre>cd LocalAI</pre><pre>docker-compose up -d - pull always</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*krtBM4n5lJRioPkQ" /></figure><p>A more detailed setup tutorial written by Tyler from Spectro Cloud, which shows the K8sGPT integration is <a href="https://itnext.io/k8sgpt-localai-unlock-kubernetes-superpowers-for-free-584790de9b65">HERE</a></p><p>What is important is choosing which model is best for the job and I want to share with you some insights after using such a workflow for a couple of weeks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GYJvqB9iNcDlSifndWR7wQ.png" /><figcaption>model comparison</figcaption></figure><p>So, all in all, I would highly recommend that you start experimenting with embedding <a href="https://ubuntu.com/blog/what-are-large-language-models-llms">LLMs</a> into your terminal workflow. There are tons of tools allowing you to do that in a secure and convenient manner on a local setup. It’s also not a super resource-hungry setup and any decent developer laptop like XPS 13, MacBookPro, … are handling it easily. If you are looking productivity boost you should definitely try it.</p><p><strong>If you would like to learn more about generative AI, large language models or open source MLOps, meet me &amp; my team during </strong><a href="https://ubuntu.com/ai/roadshow"><strong>Canonical AI Roadshow.</strong></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9489df6b9ae1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ubuntu-ai/5-tools-to-improve-your-terminal-productivity-with-llms-9489df6b9ae1">5 tools to improve your terminal productivity with LLMs</a> was originally published in <a href="https://medium.com/ubuntu-ai">Open Source AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ML models security — from MLOps to inference]]></title>
            <link>https://medium.com/ubuntu-ai/ml-models-security-from-mlops-to-inference-685fb0d01a43?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/685fb0d01a43</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[infosec]]></category>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Wed, 08 Mar 2023 09:18:03 GMT</pubDate>
            <atom:updated>2023-03-08T10:44:56.540Z</atom:updated>
            <content:encoded><![CDATA[<h3>ML models security — from MLOps to inference</h3><p>Security is an important part of any computer system. IT security is a well-described and known field. Red teams are constantly trying to find new vulnerabilities and blue teams are hardening their systems and mitigating attacks.</p><p>Applications using machine learning are no different. MLOps pipelines, inference servers, and data lakes also need to be secured, CVE patched, and properly hardened according to local regulations (FIPS, DISA-STIG, PCI-DSS, NCSC …).</p><p>However, in the field of data science, we have new attack vectors and threats that are less known, and protection against them is not yet an automated routine of every SoC on the market.</p><p>Let’s take a look at some new surprises that a red team or an APT group can present next time they target us.</p><h3>High-end attacks against ML</h3><p>As we can see in the paper <a href="https://arxiv.org/pdf/2206.08451.pdf">“I Know What You Trained Last Summer”</a>, much more interest and research are happening on the defense side, and the total volume of publications on that topic is low.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SWZdYg3ho-gcc_RW" /><figcaption>Literature on ML model security</figcaption></figure><p>As a red team member, it’s an amazing situation, as if I’m a little lucky defenders will have no idea what hit them.</p><h3>CIA triangle</h3><p>There can be different goals for the attacker. They can target confidentiality, integrity, or availability.</p><p>Confidentiality — target the training data (like model inversion) or the IP behind the model (neural network architecture and hyperparameters extraction)</p><p>Integrity — target prediction quality, like trying to increase false negatives in a credit card fraud detection system</p><p>Availability attacks — the target is to make a model irrelevant by blocking access to it or increasing its error rate to a point it is useless</p><h3>Adversarial inputs</h3><p>Whenever we use a model in an open system (like a traffic camera) or exposed in MLaaS (Machine Learning as a Service) mode we are no longer in control of the inputs. This means an adversary can freely craft any input they want to avoid detection or modify the results. It can be as simple as an “adversarial t-shirt” or complex, large-scale manipulation of Tweet content as part of an INFOOPS trying to manipulate election results.</p><p>Depending on the attacker’s access we can observe white-box attacks, where they have access to the model’s parameters, otherwise, we call it a black-box attack.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/0*MgyZmd8kesCpz-VM" /><figcaption>Adversarial T-shirt</figcaption></figure><h3>Data poisoning</h3><p>You can use training data in order to control prediction behavior, especially in the case of the models using reinforcement learning or a simple scheduler that re-trains a model with new inputs every day. It can be used in both white-box and black-box scenarios.</p><p>This can be used to alter the way how recommender systems work on social media, online stores, video hosting services, or your dating app account.</p><p>This can be achieved by sophisticated automation and tools to C&amp;C (Command and Control) an attack or just a simple “troll farm” which you hire on the dark net to input product opinions and undermine your competition manually.</p><h3>Model stealing techniques</h3><p>Model stealing is the scariest type of attack on a business. Imaging training a model to trade stocks as a hedge fund, or a model for targeted therapies as a top pharmaceutical company. You need to gather tons of data, some of it proprietary for your company, and then spent cloud costs and a year of labor in your data science team. Then you expose the model, and in 10 minutes your competition can benefit from your multi-million investment without any upfront cost because they got the model.</p><p>There are different objectives of such attacks:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sPmq2cqldbwcpb3j" /><figcaption>Model stealing objectives</figcaption></figure><p>There are many examples of how model stealing works, one example is the paper CloudLeak: <a href="https://www.ndss-symposium.org/wp-content/uploads/2020/02/24178.pdf">Large-Scale Deep Learning Models Stealing Through Adversarial Examples</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*k-Y9Rh12k9BpYtA3" /><figcaption>Model stealing process</figcaption></figure><p>the process is:</p><p>A- generate unlabeled adversarial examples as a synthetic dataset</p><p>B — query the victim model using the generated synthetic dataset</p><p>C — label adversarial examples according to the output of the victim model</p><p>D — train the local substitute model using the synthetic dataset</p><p>E — use the local substitute model for predictions. The local substitute model is expected to match the performance of the victim model</p><h3>How to defend</h3><h3>Detection</h3><p><strong>Watermarking</strong></p><p>Watermarking is a well-known technique, used ie. on photos or money. Model watermarking is a way to prove ownership of the model. You can achieve that by embedding data that are known only to you (specific parameters, weights, additional detection capabilities ie. for your brand logo). This is not protecting your models from being stolen but can help your lawyer in a court of law when you find out that your competition is using a stolen model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ppht-s_Tv7rzSV2S" /><figcaption>Watermark</figcaption></figure><p><strong>Monitoring based</strong></p><p>An LMA stack (Logging, Monitoring, Alerting), like an <a href="https://ubuntu.com/blog/lma-2-modern-observability-with-juju-by-charming-the-grafana-ecosystem">open-source COS</a> can help a lot in detecting adversarial attacks against your models. You can do it by observing traffic to your inference API and looking for anomalies or specific patterns that are used in model extraction attacks described above. Best open-source observability tools already have predefined alerting rules that you can use for that.</p><h3>Prevention</h3><p><strong>Re-training from scratch</strong></p><p>You can re-train a model from scratch with a different architecture or parameters, getting an accuracy that is close to the original model and using a blue-green deployment mechanism to direct some of the adversarial traffic to it. This will significantly decrease the performance of the knock-off model, the results I got from testing this was lowering the stolen model F1 score by 42%</p><p><strong>Differential privacy</strong></p><p>Differential Privacy protects against stealing the decision boundary of a model. Their main idea is to make outputs of all samples lying in the boundary-sensitive zone, i.e. samples that are close to the decision boundary, indistinguishable from each other. This is achieved by adding perturbations to these outputs, by a so-called boundary differential privacy layer (BDPL).</p><p><strong>Input and output perturbations</strong></p><p>The main idea is to perturb input/output probabilities to make the gradient maximally far from the original. Some papers proposed to use of the reverse sigmoid activation function as a</p><p>defense. A specific characteristic of that function is that it maps different logic values to the same probability. This leads to wrong gradient values and complicates the stealing process.</p><p><strong>Model modification</strong></p><p>Contrary to perturbing the data, which aims at reducing the precision of the behavior of stolen models, one can modify the model architecture and/or parameters. The motivation for protecting the architecture can e.g. be in scenarios when this architecture is novel and has certain advantages over existing ones. The main goal of the defender is thus not to protect one specific trained instance of this architecture (i.e. the learned model parameters) or training hyperparameters, but more in general the architecture itself, as it would prevent an attacker to apply this architecture to a different domain</p><h3>Open-source</h3><p>Automating all of the above defense mechanisms can be done using an MLOps pipeline like <a href="https://charmed-kubeflow.io/">Kubeflow</a>. The defense strategy, that can be implemented as part of your ML Ops infrastructure could look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cdYSmSejlAOP2ECB" /><figcaption>Defense strategy for ML models</figcaption></figure><p>If ML model security is a topic that interests you I would recommend starting by learning about INFOSEC in general, and in parallel trying dedicated open-source tools like <a href="https://mlsploit.github.io/">MLsploit</a>. MLsploit is the first user-friendly, cloud-based system that enables researchers and practitioners to rapidly evaluate and compare state-of-the-art adversarial attacks and defenses for machine learning (ML) models.</p><p>If you want to hear which defense mechanisms are used in the healthcare industry and how to protect patient privacy I want to invite you to my Kubecon keynote <a href="https://kccnceu2023.sched.com/">https://kccnceu2023.sched.com/</a></p><p>You can also reach me through my social media channels listed on <a href="https://www.maciejmazur.com/">https://www.maciejmazur.com/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=685fb0d01a43" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ubuntu-ai/ml-models-security-from-mlops-to-inference-685fb0d01a43">ML models security — from MLOps to inference</a> was originally published in <a href="https://medium.com/ubuntu-ai">Open Source AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Modern Warfare — AI the military space in 2023]]></title>
            <link>https://medium.com/weles-ai/modern-warfare-ai-the-military-space-in-2023-5eb963863728?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/5eb963863728</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[defense]]></category>
            <category><![CDATA[military]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[innovation]]></category>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Thu, 09 Feb 2023 18:22:23 GMT</pubDate>
            <atom:updated>2023-02-09T18:22:23.562Z</atom:updated>
            <content:encoded><![CDATA[<h3>Modern Warfare — AI the military space in 2023</h3><p>The last global conflict, as horrible as it was, brought us transistors, ball pens, radar, and many other inventions created in face of imminent danger. We are again witnessing an unstable period. In this article, I want to share my views on how the defense industry is rapidly modernizing in 2023, with the widespread adoption of state-of-the-art machine learning.</p><p>Since I combine years of experience working in tech as well as wearing a uniform I’m most interested in the impact on the defense sector, but most of this is applicable to other verticals as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XAfjyRAxdGSmiVP0KD-MhQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/fr/@michael_schiffer_design?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Michael Schiffer</a> on <a href="https://unsplash.com/photos/9CSGSq_7UrA?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><h4>Logistics and transportation</h4><p>The last few years brought us back to high-scale and high-intensity conflicts between nations. Before looking into all the science fiction weapons and Black Mirror technology let’s tackle the most overlooked area — logistics. As gen. Pershing said, <em>“Infantry wins battles, logistics win wars”.</em></p><p>Most of the armed forces are part of an alliance of some sort like NATO, BRICS, or EU. This means that any group, from a small multi-national unit to a tank battalion will contain various forms of similar, yet very different equipment. An alliance would have 3–6 different types of main tanks or fighter jets. All of them would need different ammunition, fuel, and spare parts. That’s a quartermaster’s nightmare.</p><p>However this is not a new problem, in fact, most retailers and FMCG companies already built massive sets of ML models to solve such issues like:<br>- identifying the possible impacts of supply disruptions, demand changes, or discrepancies<br>- automating administrative tasks like filing paperwork<br>- forecasting customer demand through data-driven data analytics<br>- analyzing traffic and weather conditions to optimize routes, saving time, fuel, and money<br>- utilizing smart warehouses like UK’s Ocado</p><p>We already see that the defense sector is adopting this technology, but in 2023 this is going to rapidly speed up, as more and more heavy equipment is being transported around the globe and investments in such systems have a huge ROI.</p><p>Another big push is happening on the predictive maintenance front. It’s much easier to schedule spare parts delivery when you know exactly when and where they will be needed. Last year already we had digital twins on the NVidia Omniverse platform utilized to train predictive maintenance models for advanced weapon systems to the point where the accuracy is so high that mechanics call it witchcraft.</p><h4>Intel gathering and cybersecurity</h4><p>Cyberspace is this day a real domain of conflict alongside sea, air, land, and space. Machine learning is changing the landscape here as well, both on the blue and red team side.</p><h4>Defense</h4><p>Security Operations Center — SOC, which the main job is to:<br>- maintain security monitoring tools<br>- investigate suspicious activity<br>- perform routine checks of TTPs<br>- ensure compliance<br>is constantly overworked and understaffed, which is even more true in times of conflict. SOCs are already utilizing ML-based tooling for many years, however, the upcoming year is bringing new quality into that space. The research team of SOCCRATES program published an algorithm that was developed for automatically suggesting courses of action to the SOC operator. It analyzes the structure of attack graphs generated from models of the monitored ICT infrastructure and fits defense actions into a specified available budget. This is a crucial capability to win the attrition war with your opponent.</p><p>Another interesting spot for AI-driven innovation is ISTAR which stands for intelligence, surveillance, target acquisition, and reconnaissance. It has never been easier to build a data lake with multiple data sources, easily searchable, and with real-time streaming capabilities. Now we can also fully utilize it. IMO 2023 will be a year of multimodal models, which will allow us to combine text, video surveillance, audio recording, and more into a single model. Imagine DALL·E combined with Chat-GPT.</p><h4>Offense</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mFrLNdVa8Vrfo8dOa5_MBQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@maxwbender?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Max Bender</a> on <a href="https://unsplash.com/photos/XIVDN9cxOVc?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Sometimes the best defense is a powerful and well-timed offense.<br>The big disruptor is already mentioned Chat-GPT and tons of similar models like NVidia NeMo Megatron. They were immediately taken by criminals and used to create high-quality phishing and other types of social engineering attacks. We are already preparing for to LLMs be used also in info ops, election meddling, and other similar activities performed by state actors or associated APT groups.<br>What was earlier possible only as a targeted red team operation will be used this year on a mass scale to achieve strategic and tactical goals with little to no human oversight.</p><p>Everyone is trying to weaponize AI, and there is a good reason for that. Just 24% of cybersecurity teams are fully prepared to manage an AI-related attack, according to a recent Gartner survey. What is often overlooked however is that every AI system and every ML model is also a possible attack vector, which is completely new and mostly uncovered by existing security procedures. This can be as simple as malicious attackers finding small variations in the model data inputs that can result in redirected and undesired model outputs. But also more elaborate attacks like data poisoning, transfer learning attacks, or hostile model extraction stealing years of your research work are happening more and more frequently. In my next article, I will describe a full spectrum of attacks against ML models and how a good build MLOps pipeline can help to counter that with ie. inserting noise and intentional randomness into the training data and proper tuning of the model.</p><h4>Aiming and sighting systems</h4><p>Today the conflicts are not fought by two lines of soldiers standing in front of each other on an empty field but in highly populated areas where it’s hard to distinguish friend or foe. In order to eradicate civilian casualties a lot of effort is put into enhancing the accuracy of target recognition. This is gaining importance with the growing usage of autonomous weapon systems. The main technology making an impact here is the ability to export lite machine learning models, capable of running directly on the device/weapon system, which allows it to act in near real-time.<br>Not only identification is important. In target-rich environments, it’s crucial to do correct prioritization, which will use available autonomous weapons to destroy targets that would cause the highest casualties amongst friendly troops.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n9I2uK1tCp2fCQsU0DWXCw.png" /><figcaption>Vortex NGSW-FC from Vortex</figcaption></figure><p>Another recent advancement in sighting systems is NGSW-FC. Next Generation Squad Weapon — Fire Control is an advanced optic that contains a laser range finder, and a ballistic calculator and provides the shooter with an automatic, adjusted aimpoint. This capability transforms the “average” shooter into a “marksman” capability on the first shot, making him more accurate. What happens this year is the ongoing integration of such scopes into wider battlefield control systems so that firing solutions could be based on the entire dataset of battlefield consciousness instead of a single device point of view.</p><h4>Drones</h4><p>Drones are already a common part of the landscape of any modern conflict. They are used to gather information, carry munitions and support ground elements in their operations. This year however their capabilities are going to be greatly increased by two new capabilities. Creation and control of drone swarms and multi-domain operations on network-centric battlefields.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fjx5ME7jSvR1uujxarf07A.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@tumbao1949?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">James Wainscoat</a> on <a href="https://unsplash.com/photos/b7MZ6iGIoSI?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Swarm intelligence research is very advanced now, and its military application into swarms of drones is based on nature — the birds or fish can move together in formation, while also moving individually. And so can UAVs UGVs etc. <br>Swarm intelligence is pretty simple, and based on three types of movement:<br>separate — stay a certain distance apart<br>align — move towards the destination<br>cohere — try to stay together<br>Thanks to that there is no need for a single control point or any reliance on connectivity with the operator.</p><p>Where it evolves is multi-domain operations with drones. Autonomous Multi-domain Adaptive Swarms-of-Swarms (AMASS) is a new project in development now, where DARPA’s Strategic Technology Office is seeking innovative proposals from industry sources capable of creating a common C2 language meant to protect unmanned, autonomous swarms of different types against adversary anti-access/area-denial capabilities.<br>The program aims to create a swarms-of-swarms system that will simultaneously counter multiple adversarial assets and enable warfighters to operate within the A2/AD environment. An approach used here is to utilize techniques from the IoT world, like mesh twin learning which I described in a <a href="https://www.youtube.com/watch?v=oWauywr2RGs">video in a manufacturing context</a>.</p><h4>Decision support systems</h4><p>The basics of decision support systems are nicely described by my teammate <a href="https://medium.com/ubuntu-ai/unleashing-the-power-of-ai-in-decision-support-systems-cb95c7594177">here</a> so I will focus only on the new things coming into the picture.</p><p>DSSs are commonly used now in strategic decision-making, wargames, and tabletop exercises. What we are working on right now is integrating it into real-time tactical decision-making. DSS is trained on data coming from battlefield simulators which are taking into account squad-level tactics and situations. In order to be able to perform reinforcement learning and also give this capability to a single infantryman it integrated with the TAK system which the operator can carry on his plate carrier and see hints about a mission at hand, probabilities of success of certain actions, and dynamically adapted priorities and mission parameters. All of the data from the unit is later fed back to the MLOps pipeline and used to improve the DSS iteration by iteration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*aG3qL8LEPZ62IENHYG6XMQ.jpeg" /><figcaption>Samsung model that I use in a Juggernaut case from <a href="https://juggernautcase.com/">https://juggernautcase.com/</a></figcaption></figure><h4>Enhanced combat casualty care</h4><p>War is a dangerous game, and in order to lower the risk we invest heavily in combat casualty care. There are two systems developed right now that caught my attention.</p><p>The first one is automated triage and evacuation planning. Whenever someone gets injured and needs specialized help the MEDEVAC is called. From a data science point of view, it’s a good data source being a standardized 9-line format. So automated voice recognition models are used to get the request in real time into the system when medevac is being called, and a team with proper equipment, transportation means, and security is automatically dispatched taking into account all external priorities. This allows a soldier to get help quickly in a “Golden Hour”. The first hour after the occurrence of a traumatic injury is considered the most critical for the emergency stabilization of a casualty.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*LSqWXCHWe9A_a4phb2yboQ.jpeg" /><figcaption>Rite in Rain card based on Field Manual FM 8–10–6</figcaption></figure><p>A big factor impacting survivability is also immediate care, delivered typically under TCCC — Tactical Combat Casualty Care. An issue here is that not everyone can be a trained medical professional, with all the knowledge of a world-class surgeon. But everyone can talk to a Chat-GPT-like model over the radio, describe patient conditions and get the best possible advice on how to proceed. This turns everyone with a radio into an AI-enhanced combat medic doing clinical reasoning in the same way as top doctors. And there is no limit to such calls, each and every injury gets proper attention and advice. Such systems are currently tested in real conditions and I can’t wait to publish the overall results but even an initial look shows a great life-saving potential of such an approach.</p><h4>Conclusions</h4><p>Key decision-makers on the state level understood that technological superiority is a key to controlling a modern battlefield. Widespread adoption and dynamic improvements of AI-based capabilities are already in motion and only going to speed up. The key factor is now the speed of translation of ML models, data platforms, and other tools from the civilian world into the public sector. Open source technologies are a key enabler to getting the latest and greatest software, benefiting from innovation yet still staying in control, being able to audit the source code, and trusting the automation in charge of powerful weapons.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5eb963863728" width="1" height="1" alt=""><hr><p><a href="https://medium.com/weles-ai/modern-warfare-ai-the-military-space-in-2023-5eb963863728">Modern Warfare — AI the military space in 2023</a> was originally published in <a href="https://medium.com/weles-ai">weles-ai</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Adoption for Communication Service Providers — Quick Overview]]></title>
            <link>https://medium.com/pgs-software/cloud-adoption-for-communication-service-providers-quick-overview-aad2d9abf80a?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/aad2d9abf80a</guid>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Fri, 15 May 2020 05:45:26 GMT</pubDate>
            <atom:updated>2020-05-15T05:52:07.968Z</atom:updated>
            <content:encoded><![CDATA[<h3>Cloud Adoption for Communication Service Providers — Quick Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*TUvLU4XUvXQsLY8U0W1dCg.png" /></figure><p>The current world situation proves to be a testing time for Communication Service Providers (CSP), as they are facing <strong>many new challenges</strong>. The main are:</p><ul><li>a visible slowdown in revenue and new subscribers in developed markets;</li><li>investors pressuring to reduce costs and introduce more agility;</li><li>many technology changes at once: 5G, public cloud adoption, moving workloads to the edge, increasing automation and machine learning usage.</li></ul><p>Although the above can be a real problem for many CSP’s, it’s also an opportunity to expand and improve, because there’s a way to tackle these issues — <strong>Cloud adoption</strong>.</p><h3>Cloud Adoption is Growing Across All Verticals</h3><p>Mobile Network Operators have by now established many commercial relations in the industry and thus are one of the key players in Cloud adoption. Public cloud providers are hardly a competition, if MNO’s operate as a cloud broker, which can offer end to end value and a full service with much better user experience and more personalisation.</p><p>First of all, there are many internal benefits of Cloud adoption.</p><h3>Better Cost Structure</h3><p>Shifting from the traditional CapEx heavy model to OpEx driven introduction of new services improves TCO and moves a lot of risks related to a new offering away from the operator.</p><h3>A More Flexible Infrastructure</h3><p>Which allows to achieve <strong>business agility and greatly reduces time to market new services</strong>. This is realised not only by faster and much more automated development, testing and productization, but also by opening for more 3rd party providers, who may not be experts in SS7 and Diameter but experts in IT traffic monitoring and security solutions, providing high quality solutions faster than ever seen in closed environment of MNO.</p><h3>No Vendor Dependencies</h3><p>Currently, a typical operator has tens (if not hundred) of <strong>different hardware and software vendors</strong>, providing specialised platforms for a particular network part. Fortunately, ATCA times are over, and most workloads in the back of CSP datacentres now run on standard IT architectures.</p><p>This, however, didn’t remove the problem, but simply moved it one layer up.</p><p>Now, when buying a new NFV from a certain vendor, CSP’s are tied to his NFVI, MANO etc. even if in theory they should be multi-vendor capable. On the other hand, if CSP run their vMME on AWS, they’re using all standard and familiar tools for deployment, management, monitoring. You can also observe all major vendors already cooperating with cloud providers and making their network elements ready to be run on a cloud.</p><h3>Key Opportunities</h3><p>From an external business perspective, Cloud adoption provides a few major opportunities.</p><h3>Wide MEC Adoption</h3><p>You can already hear whispers in the conference lobbies, that <em>“edge is a new cloud”</em> and I think that there is a lot of truth in that.</p><p>Currently almost <strong>every big business considers moving (at least partially) to a public Cloud </strong>— benefits are obvious. Last month, some major providers had to “ration” their services due to demand overgrowing their capacity. I foresee that edge, thanks to Multi-Access Edge Computing and wider 5G adoption, will be the next most wanted resource.</p><p>There are many new business use-cases enabled by low latency and close to the end user platform to host applications. I see a great <strong>synergy here between an MNO </strong>(experienced in RAN, 5G and general maintenance of base stations) <strong>and a public cloud provider</strong>, who brings a very efficient way of re-selling computing capacities to many users in a self-service mode. Imagine that a developer deploying his app is not choosing between Frankfurt and in London, but rather selecting a certain area where his app should operate, making it a truly distributed service.</p><h3>Becoming An E2E Solution Provider</h3><p>Mobile operators already have a huge customer base, since literally every business needs a phone and a mobile subscription. In addition, there is a huge network of physical service points in every city in the country. This is a much more convenient way of <strong>delivering E2E IT services at scale</strong>, and also very attractive for public cloud providers, extending their reach even further.</p><p>User experience is much greater if customers can go to one place, buy phones, office software subscription, cloud subscription, automated call centres on the cloud and a sales prediction engine, put all of this on one bill (and in case of trouble, have a single point of contact). Such an approach would also allow to introduce industry specific solution ie. furniture factory IT, containing all necessary services, tailored to business needs of a targeted vertical.</p><h3>Repackaging Existing Capabilities</h3><p>MNO’s have a lot of software-based services they provide to subscribers, from simple corporate voice VPN’s to IoT platforms, dedicated for oil pipelines monitoring. All of them are very valuable pieces of IP. Migrating them to the cloud is not only giving typical benefits mentioned in the beginning of this post but also allows to <strong>package it and provide in a Cloud marketplace available to others</strong>.</p><p>A small operator from a developing country would not be able to invest into building such solutions but might use a stable and proven product from a major player available for him in one click. This generates a new revenue stream, which could cover the migration costs and make migration “free”.</p><h3>Summary</h3><p>Although the list of challenges can seem overwhelming for a single organisation, Cloud adoption may solve most of the issues and provide some added value.</p><p>Ultimately, current issues very often work as a kick-start for changes, which will bring benefits for many years to come.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aad2d9abf80a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/pgs-software/cloud-adoption-for-communication-service-providers-quick-overview-aad2d9abf80a">Cloud Adoption for Communication Service Providers — Quick Overview</a> was originally published in <a href="https://medium.com/pgs-software">PGS Software</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Although the mobile industry is booming, major Mobile Network Operators didn’t have that many…]]></title>
            <link>https://medium.com/pgs-software/although-the-mobile-industry-is-booming-major-mobile-network-operators-didnt-have-that-many-64a74a4b1626?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/64a74a4b1626</guid>
            <category><![CDATA[son]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[5g]]></category>
            <category><![CDATA[mlops]]></category>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Thu, 16 Apr 2020 14:41:41 GMT</pubDate>
            <atom:updated>2020-04-20T11:15:17.331Z</atom:updated>
            <content:encoded><![CDATA[<h3>Machine Learning for 5G — An Opportunity for Mobile Network Operators And Software Houses To Collaborate</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iZ-stajAPTmKQe3-0Ldf1g.jpeg" /></figure><p>Although the mobile industry is booming, major Mobile Network Operators didn’t have that many opportunities to collaborate with Software Houses.</p><p>Until now.</p><p>Most software used by Mobile Network Operators (MNO), from <strong>Customer Experience Management</strong> to <strong>Core Network Operations</strong>, is developed as part of a bigger deal with <strong>Radio Access Network (RAN)</strong>vendors and is a closed ecosystem. This leads to a vendor lock-in (which is not that terrible, as there are only several vendors available anyway) but the bigger issue is that such a vendor needs to build his software to match the requirements of almost each telecom operator on the planet. Such requirements are many times contradictory, which leads to multiple compromises and roadmap delays.</p><p>Thus, tailor-made solutions are needed, as they <strong>simplify the daily workflow and reduce OPEX</strong>.</p><h3>A New Approach To Developing Software</h3><p>With the introduction of 5G comes a huge mindset switch. MNOs quickly noticed business benefits from offloading less critical services out of their own data centre to the cloud and are opening themselves to modern IT practices, DevOps, micro-services, etc. As a result, software developers from typical Cloud development shops are <strong>now exploring the world of mobile networks</strong>, and Network Engineers are getting to know how to better manage and automate the increasing complexity coming from yet another RAN technology.</p><p>In this article, I’ll share my experience related to machine learning and other disciplines hidden under the umbrella of data science, where I saw biggest benefits from MNO-Software House collaborations.</p><p>Here are three examples from my engagements with telecom operators from Q1 2020:</p><h3>1. So much more data!</h3><p>The fuel for every machine learning engineer is good and reliable data.</p><p>5G is introduces <strong>many new data sources</strong>, significantly increasing data volumes and speed in which they are generated. For some MNO’s, implementing this might be a challenge, but for an experience data engineer it is bread and butter. By ensuring proper data collection, ETL, and data cleaning with tools like Scala, Spark, Snowflake, Kafka and others, data engineers can quickly harness huge amounts of data coming from the network.</p><p>Unfortunately, data engineering is not enough. To get insights out of this data — which can be used either to optimize the service or increase customer satisfaction — additional help is needed. At initial engagements I usually see passionate network engineers turned into data scientists working in their local notebooks — however, this usually means reinventing the wheel. What usually helps is introducing the <strong>ML Ops concept</strong> with such a way of working.</p><p>If you are not familiar with this approach, I would recommend you start by reading <a href="https://martinfowler.com/articles/cd4ml.html">Martin Fowler</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aWr4ZzvS2E2lCDil.png" /></figure><p><strong><em>The End-to-End CD4ML Process</em></strong><em> — source: martinfowler.com</em></p><p>Implementing ML Ops allowed one of the operators in APAC to triple the number of analysis their data science team was able to produce — and such a project takes only 2 months to complete.</p><h3>2. Self-Organising Networks (SON)</h3><p>In the era of 5G, <strong>Self-Organising Networks are even more important than earlier</strong>, as there are more sites, antennas, RAN features which need optimising.</p><p>Although equipment vendors provide SON solutions dedicated to their network elements, there’s still a big need for network level Self Organising Networks, which may be described as <strong>SON Coordinator components</strong>. They gather data from all vendor specific systems and coordinate them to achieve optimal configuration from the entire network’s perspective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c3oBNv9J7D0jKIqM.png" /></figure><p><strong><em>Increasing network complexity</em></strong></p><p>Imagine that your d-SON (distributed SON) function running on a base station from VendorA operates on a neighbour cell from VendorB needing handover optimisation triggered by its c-SON (centralized SON). Such an approach allows things like — scenario based SON, user group or device group-based SON and integrations of network management to CEM to propose optimisation automatically based on user complaints. Such advanced SON features require a <strong>big amount of computing power and utilising public cloud ML’s</strong> like AWS SageMaker lowered the OPEX of SON coordination system 4 times.</p><h3>3. Smart Tokenisation</h3><p>One might now ask if any data is safe when offloaded to the public Cloud. Luckily, there’s a IT solution for that — <strong>smart tokenisation</strong>. It resembles storing raw data in an encrypted place. The tokenisation function residing on top can generate anonymized yet still meaningful data, to mask and protect original and sensitive content.</p><p>For example:</p><ul><li><em>Name and Surname</em> — real looking — but fake</li><li><em>Address</em> — valid only in terms of general location</li><li><em>Email</em> — fake but with a valid DNS — alias</li><li><em>MSISDN</em> — totally fake</li></ul><p>Tokenisation can store unique checksums, IDs and encryption keys in separately secured places and monitor possible changes. It can catch (near-real-time) any attempt of data tempering. All information is rotated quickly (with re-generated new fake data). Such an approach enables:</p><ul><li><strong>Data tracking</strong> — near-real-time monitoring and auditability, showing each change, source and difference compared to raw information. It provides on-the-go security of the data platforms.</li><li><strong>Data sharing</strong> — inside and among organisations — any sensitive information is masked or aliased, having meaning only to the initial data keeper.</li><li><strong>Subscriber protection</strong> — real-time monitoring over subscriber and aliases allows implementing additional security mechanisms like content filtering, attack protections, leakage blockers, etc.</li></ul><p>Such an approach allowed to expose more valuable data via 5G NEF and monetise insights coming from network traffic, keeping subscriber privacy. This can be used even now to share population movements data to researchers fighting COVID-19 without privacy risks.</p><h3>Summary</h3><p>This concludes our list of practical examples from the last quarter. As you can see, combining the broad telecom network knowledge of operator engineers and agile IT of Software Houses can clearly be beneficial to bringing the world closer and fully interconnected.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64a74a4b1626" width="1" height="1" alt=""><hr><p><a href="https://medium.com/pgs-software/although-the-mobile-industry-is-booming-major-mobile-network-operators-didnt-have-that-many-64a74a4b1626">Although the mobile industry is booming, major Mobile Network Operators didn’t have that many…</a> was originally published in <a href="https://medium.com/pgs-software">PGS Software</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[More Effective Machine Learning Production with MLOps]]></title>
            <link>https://medium.com/pgs-software/more-effective-machine-learning-production-with-mlops-78c4036beee4?source=rss-3949e5403b98------2</link>
            <guid isPermaLink="false">https://medium.com/p/78c4036beee4</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[business-software]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[machine-lear]]></category>
            <dc:creator><![CDATA[Maciej Mazur]]></dc:creator>
            <pubDate>Thu, 02 Jan 2020 08:10:18 GMT</pubDate>
            <atom:updated>2020-01-02T08:10:18.278Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GzyDh2CrRYth5ua6w81IpA.jpeg" /></figure><p>Machine Learning is slowly starting to mature as a field, but many businesses are still facing tough upfront challenges. While there are a huge (and increasing) amount of new algorithms, neural network architectures and libraries, practical deployments are still in the toddler phase.</p><p>However, there’s plenty of new hope for businesses looking to deploy Machine Learning. Just as DevOps revolutionised how software is developed and delivered, I want to showcase a similar solution here: the aptly named MLOps.</p><h3>The Current State of ML</h3><p>Machine Learning is a very new technology — one that is already bringing benefits for many — but, like any technology, there’s a familiar development story…</p><p>The current state of Machine Learning projects reminds me of the early days in software development — around 10–15 years ago — where we started introducing proper development processes. Looking back, I feel embarrassed about how we delivered software projects those days. As a software development community, we’ve made huge improvements regarding how we productise our code, use a GIT repository, CI/CD pipelines, proper testing on many different levels (unit, integration, E2E, mutation …) etc. In short, there have been a lot of key achievements made already, but still more to go.</p><p>It’s also very exciting to work in Data Science these days, as I’m sure we can achieve great improvements and push our domain forward using MLOps.</p><h3>What is MLOps?</h3><p>MLOps, which means “Machine Learning Operations” is a development practice similar to DevOps, but built from the ground up for Machine Learning best practices. Just as <a href="https://www.pgs-soft.com/blog/how-devops-can-redefine-your-it-strategy/">DevOps</a> looks to provide regular, shorter releases, so too does MLOps look to enhance every step of the ML development lifecycle.</p><p>In fact, many factors from DevOps directly apply here. Most importantly, a focus on Continuous Integration/Continuous Delivery is applied directly to model generation, while regular deployment, diagnostics and further training can also be done on a frequent process, rather than waiting for one large upload at much slower intervals.</p><h3>MLOps &amp; Delivering ML-based Value</h3><p>Like any solution, Machine Learning (ML) is measured by its success and value.</p><p>Some companies are asking about the newest and fanciest things, such as automated model retraining. The typical initial reaction to such a request is to take a step back and start with questions such as „can you easily repeat what your data scientist did yesterday ?”.</p><p>We somehow naturally expect that, if we have code versioning, continuous integration etc in our software projects, it’s automatically available for our models. This is one of the biggest misconceptions: that all we do is “just” train a model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/0*yPlB0WsX92v38h5z.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/0*c9cYwb92mvBVDd0p.png" /></figure><p>Expectations are one thing, but the reality is often different. A more popular real life scenario is talking to the CTO and hearing something along the lines of: “Hey, we hired a bunch of data scientist, they play all day with Jupyter notebooks on their local machines and, when it’s time to deploy it, we have no idea where the code is. Mostly, it’s not even checked in. They claim to do so much great stuff but we cannot get any business benefits out of it.”</p><p>If you are considering starting with MLOps I would first suggest to evaluating your level of maturity regarding both how you work on your models and how they are deployed to production environments.</p><h3>Key Machine Learning Values</h3><p>So, what do we want to achieve from Machine Learning? After all, this is the end goal that MLOps is striving to achieve more efficiently. For most businesses, we can determine values at three key levels or stages:</p><p><strong>Repeatability </strong>— this means you have your work on the model automated enough that you can always repeat any experiment your data science team already performed.</p><p><strong>Productisation </strong>— at this level, you should be able to automatically push the model to production, having automated not only the deployment, but also canary deployments, testing, fallbacks, monitoring etc…</p><p><strong>Business Benefits</strong> — at this level, you should be able to measure the influence of each and every model version in temrs of business metrics. For example, when you deploy a new recommendation engine to a retail website, or to a small user group, you need to automatically see on the model repository that the conversion rate or sales figures were influenced by X% with this deployment.</p><p>We can find similar examples of all industrial corners:</p><ul><li>Does your manufacturing or production facility need to be optimised across numerous parameters? Automation can help manage large workloads that manual human activity can’t keep up with (and for the largest of operations across numerous plants, there’s <a href="https://www.pgs-soft.com/white-paper/mesh-twin-learning/">Mesh Twin Learning)</a></li><li>Is your financial company trying to stay on top of online fraud? Automation is important for learning about new challenges, updating to newer versions and staying ahead of the field. What’s more, you need to know if it worked!</li><li>And so on. You get the idea…</li></ul><p>In order to enable this, you also should consider both data management and the analysis of new datasets becoming available in the system. It’s good to be able to know if any new data is similar to what we already have, or they have different distribution, and if it could be worthwhile to re-train the model with it.</p><h3>Personas in the Process</h3><p>One key challenge in the MLOps process comes from the many positions and hats involved in it. You need to be able to meet the needs of your Data Engineer, making sure that you can get, clean, version and maintain the datasets, which gets even more tricky with real-time event based systems. And that’s just the first person!</p><p>Next on the list is your Data Scientist, who sometimes is really more of a scientist than an engineer, to enable him or her to run experiments and test many hypotheses. Finally, you need someone to build the model into your product, making sure that your experiment results are also the same in the real world. This is where the ML Engineer comes in.</p><p>With this many people involved, the process needs to be as efficient and streamlined as possible, speaking of which…</p><h3>Creating the MLOps Pipeline</h3><p>There is a lot of existing tooling that can help with building an efficient MLOps pipeline. You can go the open source way, for example, and use tools such as Airflow, Kubeflow and Jenkins, paired with a few custom scripts of your own.</p><p>Likewise, all the major Cloud providers have their own set of DevOps tools, which can be extended by some open source components to build a full MLOps solution.</p><p>However, for this article, I want to showcase another option, which is to go with TensorFlow TFX: an end-to-end platform for deploying ML pipelines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/0*gcvGuHe2nrPvXIxU.png" /></figure><p>Its key strength is that, besides running on Google Cloud Platform (GCP) , you can use Apache Airflow or Kubeflow as runtime .</p><p>Another highly appreciated aspect here is the metadata store concept, which contains:</p><ul><li>Trained models (Models, Data &amp; Evaluation metrics)</li><li>Execution records (runtime configurations, as well as inputs + outputs)</li><li>Lineage tracking across all executions (ie. To recurse back to all inputs of a specific artifact)</li></ul><p>Also, on the picture above, you can see how every atomic operation is handled. You have a driver and publisher (if needed, they can be overwritten) which handle the fetching and storing of data in the metadata store, as well as an executor for writing the business logic of a certain step.</p><p>This enables the following use-cases:</p><ul><li>visualising the lineage of a specific model</li><li>visualising a data specific model that was trained on a certain run (with sliced evaluation metrics, which you can see below)</li><li>comparing multiple model runs</li><li>comparing data statistics for different models</li><li>carry-over states from models run some time ago</li><li>reusing previously computed outputs.</li></ul><p>My favourite is sliced evaluation metrics, where you can analyse how certain parts of the dataset are influencing your results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xnq_PGd_V3rQ2WOw.png" /></figure><p>Another one that we use a lot, especially in the early stages of a project, is the possibility to reuse previously computed outputs (as they are stored in the metadata store). With this approach, when you have a training job with multiple steps, typically taking several weeks in total, you don’t need to start over from scratch if anything fails.</p><h3>Summary</h3><p>So, where is MLOps most useful? If you are a single researcher working on your thesis or some small pet project, you don’t need to complicate your life with MLOps. If, however, you work with any kind of production system, I highly recommend you try it.</p><p>If you need any help in choosing a proper solution for you or you already decided for one and have some issues implementing it please <a href="https://www.pgs-soft.com/contact-us/">contact us</a>.</p><h3>Business Perspective</h3><blockquote>Machine Learning doesn’t need to be a long development cycle. MLOps combines the best parts of DevOps and adapts it to the development, release and training of ML models to help gain immediate benefits and quicker releases for all important Machine Learning algorithms — all of which enables businesses to start implementing such advanced solutions faster than ever.</blockquote><p><em>Originally published at </em><a href="https://www.pgs-soft.com/blog/more-effective-machine-learning-production-with-mlops/"><em>https://www.pgs-soft.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78c4036beee4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/pgs-software/more-effective-machine-learning-production-with-mlops-78c4036beee4">More Effective Machine Learning Production with MLOps</a> was originally published in <a href="https://medium.com/pgs-software">PGS Software</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>