<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Maven Wave Data Science Blog - Medium]]></title>
        <description><![CDATA[Maven Wave is a unique, one-stop-shop offering industry-specific go-to-market strategy to accelerate the global enterprise’s migration to the cloud. - Medium]]></description>
        <link>https://medium.com/maven-wave?source=rss----96fef1a9a525---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Maven Wave Data Science Blog - Medium</title>
            <link>https://medium.com/maven-wave?source=rss----96fef1a9a525---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 11:36:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/maven-wave" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Explainable AI (for image classification), on Vertex AI]]></title>
            <link>https://medium.com/maven-wave/explainable-ai-for-image-classification-on-vertex-ai-5e72dd01e77?source=rss----96fef1a9a525---4</link>
            <guid isPermaLink="false">https://medium.com/p/5e72dd01e77</guid>
            <category><![CDATA[fire-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[explainable-ai]]></category>
            <dc:creator><![CDATA[Anurag Bhatia]]></dc:creator>
            <pubDate>Wed, 01 Feb 2023 16:39:06 GMT</pubDate>
            <atom:updated>2023-02-16T03:30:13.014Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>What is XAI?</strong>: Explainable AI is a subset of machine learning where the focus is more on understanding what goes on under the hood when a machine (or model) learns something, rather than being obsessed with getting better performance metrics even if it involves increasing model complexity several fold.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/915/0*qIlH8-h_ze4Vz5Hg.png" /><figcaption>Image credit: <a href="https://medium.com/analytics-vidhya/explainable-ai-what-is-it-why-is-it-important-41f062207235">Analytics Vidhya</a></figcaption></figure><p><strong>Why should we care?</strong>:</p><ol><li>In <em>heavily-regulated industries</em> like finance and healthcare, it’s not just good enough to have good model performance metrics. Since it involves money — or even more so if it’s a matter of life and death — we also need to be in a position to interpret and explain the model behavior to other stakeholders (e.g. business teams, regulators, etc.).</li><li><em>Troubleshooting</em>: What if the model performance metrics are not good enough? It can be super helpful if we are able to a) get underneath the model b) dig deeper into how the model is interpreting the annotated data, and finally c) does that align with what we assumed the model would learn? That is where model explainability comes in.</li></ol><p><strong>Levels of model explainability</strong>: There are two different ways in which model inference results can be explained. The <strong>global-level</strong> (sometimes also referred to as the model level) is a summary of which features carry a higher weightage vis-a-vis others. This is useful information if the dataset happens to be of a structured, tabular format. e.g. a chart showing measured importance of each column..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*ZqLJ_iFch5wezn4N.png" /><figcaption>Image credit: <a href="https://www.scikit-yb.org/en/latest/api/model_selection/importances.html#feature-importances">yellowbrick</a></figcaption></figure><p>It’s a <strong>generic</strong> summary across all predictions from a trained model, while the <strong>instance-level</strong> provides details <strong>specific</strong> to each prediction from that model. e.g. In an image-classification problem, the instance-level explainability can narrow down to the specific pixels in an image which decide which class is to be assigned to that image. By definition, such pixel info for an image will be completely independent from that of the next image.</p><p>Though LIME and KernelSHAP are also used to interpret the results of image models, we are primarily going to focus on the ‘<strong>Integrated Gradients’ (IG) and ‘Explanation with Ranked Area Integrals’ (XRAI) methods</strong> here.</p><p>To begin with <strong>IG</strong>, a <em>baseline image</em> is created which usually consists of all pixels as 0s, all of them as 1s, or random values between 0 and 1. In a series of steps thereafter, each pixel value is <em>changed</em> from the one in the baseline image to its actual counterpart value in the actual image. And for each such step/change, a <em>gradient</em> is calculated. These gradients are <em>integrated</em> and the pixels with the highest ones are superimposed on top of the actual image, to highlight which specific <strong>pixels</strong> in an image were considered most relevant to decide the appropriate class assigned to that image. i.e. The most influential image pixels (as decided by the model) are given the most weightage and depicted accordingly. e.g.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*a551eU1MoRSf8gENeSRo_Q.png" /><figcaption>Image: <a href="https://github.com/CVxTz/IntegratedGradientsPytorch">https://github.com/CVxTz/IntegratedGradientsPytorch</a></figcaption></figure><p>By definition, that implies that two of the <strong>most important factors in IG</strong> method are:</p><ol><li>the <strong>baseline</strong> image (pixel values) chosen. Reason: The model explanation is relative to the baseline. Hence, one option is to decide baseline pixel values based on the usual type of images in our image classification problem. e.g. random values for X-ray type images (where it’s safe to assume that blackish regions represent something very specific i.e. tissue in this case).</li><li><strong>num-integral-steps</strong> i.e. the number of steps involved between the baseline image and the input image for calculating numerical integration mentioned above. Higher this value, usually higher is the model explainability. The flipside, of course, is that the operation becomes more computationally intensive.</li></ol><p><strong>XRAI</strong>: Weights of the trained model are used to further train a network used for interpretation. The goal here is to <em>use existing, relatively known and simpler preprocessing steps (functions) to arrive at an approximation</em> which is (hopefully) not too different from the one used in the originally trained model. For each patch in the image, the pixel-level attributions are integrated and <em>patches are clubbed into </em><strong>regions</strong> on the basis of the values of their integrated gradients. Regions are removed one by one and trained model is invoked multiple times, to ascertain the importance of each region while inferencing.</p><p>Regarding comparisons between IG and XRAI, their respective common use-cases and when to choose one over the other, here is a pretty concise summary ..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/808/1*EHYI8j17qGM4SvDLasAXXQ.png" /><figcaption>Source: <a href="https://cloud.google.com/vertex-ai/docs/explainable-ai/overview#compare-methods">Google Cloud</a></figcaption></figure><p>Alright. Enough of theory. Time to dive into the code. We have chosen fire detection as our objective and it’s a binary classification problem since our manually curated dataset has only two classes: Fire and No-Fire. For the purpose of this blog, we assume we have already trained a model using keras and saved its checkpoints in Cloud Storage (GCS) and are now interested in knowing more about why our trained model is behaving the way it is. We start by <strong>loading our checkpoints</strong> and mentioning a few other details on what the existing trained model expects in terms of input image (for getting prediction).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/522/1*tIbTAKZUrL1gLkB7TpkNrw.png" /></figure><p>Explainability methods rely on their ability to 1) create multiple (perturbed) versions of the original input image and 2) invoke the trained model and get predictions for each of those image versions. Hence, we’ll <strong>create model signatures</strong> to get this done. The first one is to perform the usual <em>preprocessing</em> steps in case of an image file e.g. read jpg file from GCS, get the bytes, apply padding, convert it to a tensor etc..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*kRQTYoxTgYg0-jUWfZAg8Q.png" /></figure><p>We create a second one which takes the output of the first one as an input, <em>invokes the model</em>, gets the class having the highest probability and assigns the label to that input image accordingly..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*PwWTRPsTPMcNuJjOlPy1eA.png" /></figure><p>We then stitch together everything we have done so far. Summary: take our existing trained model in GCS as the starting point, create model signatures required to make the model compatible with requirements of explainable-AI, and finally save it (locally, for now).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*87o1YLwOH08rbpDE_B20Ww.png" /></figure><p>Until now, we have adjusted our trained model for explainable-AI, but have not explicitly mentioned <em>which method</em> to be used for it and values of <em>required arguments</em> (e.g. baseline pixels to be used, number of steps involved etc.), but now is the right time to do so. We do it in the form of a metadata file (in json format) which is created programmatically through an inbuilt method for this purpose in Explainable-AI SDK..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/1*vlf1AkK1IZ0qY_lFVrNJSA.png" /></figure><p>We write a custom shell script to <em>deploy</em> our model on Vertex AI. It involves a) creating a model b) create a version of that model and finally c) deploy it on a Vertex AI endpoint..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*PISfyfxjsdjkhoz25x5kNQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZADQ1qWyp661MOreDBk6BQ.png" /></figure><p>And finally, we prepare a JSON payload <em>request</em> and save the <em>response</em> back in another JSON file..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*LAMj86skAcVwEo-vLl-sHQ.png" /><figcaption>request json</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*QQIyPO13XuzfAK9a5Ae3Vw.png" /></figure><p>Finally, we use matplotlib to <strong>visualize</strong> the results of the model explainability we had set out to achieve in the first place. Here are the results..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*5_MGbrQc3J1DRt3mEial3w.png" /></figure><p><strong>How to interpret this?</strong>: Results for <em>images 2 &amp; 3</em> (from left) look fine, since maximum importance is being given to that part of the image where the burning flame is the strongest. Results for <em>images 4 &amp; 5</em> have results which are mixed. Though they do rightly focus on the bright flame areas, they also include parts which they shouldn’t have. e.g. portion of trees and individuals in images 4 and 5 respectively. Most importantly, the result for <em>image 1</em> certainly suggests <strong>something is not right</strong>. The upper half of the electricity tower seems to be getting the most attention, rather than the fire engulfing the field/hill elsewhere in the picture.</p><p><strong>What exactly is the value-add?</strong>: Model explainability can guide us towards the next steps ahead. e.g. Given our results in the image above, it suggests that even if the model prediction is correct (as is the case with all 5 input images above), the model may have learnt something very different from what we thought it would. Some of our options, going forward, are:</p><ul><li>Check whether there are lot of images with electricity tower, categorized as “Fire”. In that case, we might have inadvertently ended up inducing a <strong>possible bias</strong> in our model.</li><li>Compare the difference after increasing value of <em>‘num-integral-steps’</em></li><li>Check explainability results for 1) images of “No Fire” label as well 2) images of “Fire” label but those which show something in <em>daylight </em>(rather than just those at night, as is the case in our rather limited sample of 5 input images chosen above)</li><li>Add more images for model training, preferably in different conditions.</li><li>Do some more data <em>augmentation</em> before/while training.</li></ul><p>Hope this was helpful in explaining the concept of Explainable-AI and its implementation in a specific scenario. For more details about this fire-detection project, take a look at my <a href="https://github.com/the-data-guy/Computer_Vision">github repository</a>.</p><p>Happy model explaining..</p><p><strong>References</strong>: The biggest inspiration for this blog has been the awesome <a href="https://www.oreilly.com/library/view/practical-machine-learning/9781098102357/">computer vision book by Valliappa Lakshmanan, Martin Görner, and Ryan Gillard</a>. I found their code snippets super helpful and applied them on an image classification problem for an entirely different use-case and dataset. i.e. fire-detection.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e72dd01e77" width="1" height="1" alt=""><hr><p><a href="https://medium.com/maven-wave/explainable-ai-for-image-classification-on-vertex-ai-5e72dd01e77">Explainable AI (for image classification), on Vertex AI</a> was originally published in <a href="https://medium.com/maven-wave">Maven Wave Data Science Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Operations Research Should Join the MLOps Party]]></title>
            <link>https://medium.com/maven-wave/operations-research-should-join-the-mlops-party-21e7ab184a1a?source=rss----96fef1a9a525---4</link>
            <guid isPermaLink="false">https://medium.com/p/21e7ab184a1a</guid>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[operations-research]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Daniel Dowler]]></dc:creator>
            <pubDate>Fri, 06 Aug 2021 18:08:01 GMT</pubDate>
            <atom:updated>2021-08-06T18:10:21.945Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yl_GEs5CPRyCcrmPHPe9YQ.jpeg" /></figure><p><strong>Widening the MLOps Umbrella</strong></p><p>Data science (DS) and operations research (OR) are both founded on the application of mathematical logic and optimization to business problems. While the core models and applications for these two fields do differ, their underlying workflows for obtaining answers share a very uncanny resemblance. Both disciplines collect and preprocess data, and they create mathematical models. The models are evaluated for performance, and passing models may be deployed to generate outputs for downstream users. For both DS and OR models, the process is both cyclical and iterative, as changes in the business landscape require continual reassessment, and as research insights lead to better approaches.</p><p>The DS community is currently in the middle of an operations revolution, which has rapidly enhanced practitioners’ ability to deliver business value. The high level similarity in DS and OR workflows means that lessons learned in DS operationalization can also be applied to OR applications, resulting in large additional business value gains.</p><p><strong>A Quick Operations Research Review</strong></p><p>Today, the OR acronym isn’t as widely known as artificial intelligence (AI) or machine learning (ML). Many people, including many data scientists, don’t understand how widely OR is applied, or how core OR is to modern business use cases.</p><p>OR debuted well before the digital revolution, proving its value as far back as World War II. In peacetime, companies found that they could use OR to greatly improve business processes through the use of mathematical optimization and logic.</p><p>The OR field is large and well established. Most large companies have personnel dedicated to OR, sometimes under the names of management science, or decision science. OR is known for reliably solving problems in financial engineering, industrial engineering, management operations, and supply chain management. If you are envisioning “core decisions” and “big business outcomes,” then you are on the right track.</p><p>Many well-known universities have OR programs, and they are increasingly embracing DS and ML as part of their curriculums. A few examples include:</p><ul><li><a href="https://mitsloan.mit.edu/faculty/academic-groups/operations-research-statistics/about-us">MIT Management Sloan School: Operations Research and Statistics</a></li><li><a href="https://engineering.princeton.edu/departments/operations-research-and-financial-engineering">Princeton: Operations Research and Financial Engineering</a></li><li><a href="https://www.orie.cornell.edu/orie">Cornell Engineering: Operations Research and Information Engineering</a></li><li><a href="https://www.cmu.edu/tepper/programs/phd/program/operations-research/">Carnegie Mellon, Tepper School of Business: Ph.D. Program in Operations Research</a></li></ul><p>At the time of writing, each of these websites had a reference to DS or ML on their home pages, underscoring the natural relationship between the disciplines.</p><p>In 2021, US News and World Report listed <a href="https://money.usnews.com/careers/best-jobs/operations-research-analyst">Operations Research Analyst</a> at #5 for business jobs, and #13 for best STEM jobs, with a median salary of $84,810, a 1.1% unemployment rate, and 26,100 jobs. In comparison, <a href="https://money.usnews.com/careers/best-jobs/data-scientist">Data Scientist</a> is listed at #2 for best tech jobs, and #6 for best STEM jobs, with a median salary of $94,280, a 3.5% unemployment rate, and 10,300 jobs. The reports suggest that, at least by labor, the OR industry is more than double the size of its DS counterpart, with less pay, but also less unemployment.</p><p><strong>MLOps is Supercharging Data Science</strong></p><p>In contrast to OR, the DS field got its main start when large tech companies experimented with better ways to generate value from patterns in their large amounts of data. Before DS, a lot of data research happened slowly and more independent from the business environment. For example, data samples from a company problem might be shared with an academic institution in the hopes of gaining insights many months in the future.</p><p>The early DS innovators were highly motivated to solve practical problems within the business environment. The new approach took advantage of data that was bigger, better, and more timely, and because it was in-house, it benefited from higher quality communication. These early tech pioneers also had close proximity to key advances in data processing and programming technologies, which also happened to coincide with key advances in optimization and pattern recognition theory.</p><p>The DS field is still actively developing. Leaders in the community have moved from small modeling experiments on laptops, to huge modeling workflows in production cloud environments. There are still many businesses struggling with this type of transition, however, the opportunities and financial incentives prove to be strong motivators.</p><p>The tools and methods of operationalizing DS models have coalesced under the MLOps name — short for machine learning operations. Many excellent tools now exist for making DS efficient, consistent, and dependable for business. Feature management, training with cloud resources, keeping track of model versions and performance — these can all be automated and orchestrated using MLOps tools in the cloud.</p><p>A key part of my work on the Maven Wave DS team is helping clients to take research-oriented DS initiatives, and transform them into more viable production-ready, profit-generating, business solutions. Companies often struggle with their high DS spend to low DS output ratios. My team applies MLOps best practices and technology in ways that reduce processing time, improve model selection, allow less room for error, and make deployment easier. We also help DS teams standardize and streamline their processes, so that they are easier to maintain, manage, and monitor. We’ve seen MLOps supercharge DS solutions, making them more accessible and valuable to their enterprises.</p><p><strong>Connecting the Dots Back to Operations Research</strong></p><p>The huge advantages of modern MLOps approaches are now discussed regularly within the data science community. Many companies are currently moving to add cloud-based MLOps pipelines to their workflows. The pipelines and tools have catered specifically to DS models and applications. However, due to the similar underlying structure, many of these ideas and tools stand ready to benefit OR workflows as well. Granted, some tools must be adapted, but we can make this happen, and it is a small price to pay for the abilities on the other side.</p><p>I recently had the opportunity to lead a modern MLOps-style engagement on an OR problem for a large enterprise health care client. The company, a trend-setter in its industry, still used traditional workflows to run a massive linear programming model for drug price optimization. The existing solution took many hours to run, and it was plagued by manual configuration steps with the potential for human error. The process for obtaining price outcomes was unnecessarily variable and complex, and it required a lot of maintenance and review time from senior OR practitioners before business results could be obtained.</p><p>My team used MLOps practices to consolidate and automate many of the steps in the OR workflow. For this client’s case, we found a way to use Kubeflow and Google’s AI Platform — tools originally built for doing ML work — to create linear programming constraints on parallel distributed nodes. The linear programming model itself was constructed using PulP, which is a popular library for OR model building. In the end, we successfully married the MLOps approach to the OR application for a power set of results.</p><p>We built a pipeline that ran in less than 10 minutes; it accepted a few essential parameters at runtime, and it finished by saving a model and writing results to BigQuery for easy reporting and analysis across runs. The pipeline scaled for parallel on-demand runs in the cloud, and it released resources upon completion. Our solution meant lower IT costs, fewer manual errors, standardized examples for future work, as well as more timely and accurate results for our client’s customers.</p><p>Many enterprise OR teams have worked with traditional on-prem platforms and toolsets for years. Some of these teams are undoubtedly still attached to their legacy systems. However, as the cost to switch to more modern approaches goes down, and as the competition to do so heats up, I predict there will be higher pressure on OR teams to move their solutions towards more agile, cloud-based workflows. Some OR teams have already experimented with such solutions, however, the field stands to benefit from the standardization that an MLOps-style approach provides.</p><p><strong>An Invitation to the MLOps Party</strong></p><p>The opportunity is here for enterprises who wish to move their OR workflows from the traditional paradigm with tedious work and longer wait times, to a more streamlined and standardized paradigm. MLOps tools originally developed for data science workflows can revolutionize OR workflows as well. Moreover, the lessons learned solving data science problems, mean the OR industry stands to reap high returns for low risk by emulating the MLOps approach.</p><p>Maven Wave has experience migrating enterprise on-prem OR applications to cloud-based pipelines. I invite companies employing OR optimization techniques to consider joining the MLOps party, and I believe there are great rewards waiting for you here.</p><p>The Maven Wave data science team regularly strategizes to maximize impact. If you are interested in working with us to build your future world-class implementations, please <a href="https://www.mavenwave.com/contact/">reach out</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21e7ab184a1a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/maven-wave/operations-research-should-join-the-mlops-party-21e7ab184a1a">Operations Research Should Join the MLOps Party</a> was originally published in <a href="https://medium.com/maven-wave">Maven Wave Data Science Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>