<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mikhail Chrestkha on Medium]]></title>
        <description><![CDATA[Stories by Mikhail Chrestkha on Medium]]></description>
        <link>https://medium.com/@mchrestkha?source=rss-9deb2fadae9e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*lAO9TP2X-9X8YVSfd1PADw.jpeg</url>
            <title>Stories by Mikhail Chrestkha on Medium</title>
            <link>https://medium.com/@mchrestkha?source=rss-9deb2fadae9e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:44:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mchrestkha/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Level Up Your AI Strategy: From Models to Applications and Beyond*]]></title>
            <link>https://mchrestkha.medium.com/level-up-your-ai-strategy-from-models-to-applications-and-beyond-ee9a16322235?source=rss-9deb2fadae9e------2</link>
            <guid isPermaLink="false">https://medium.com/p/ee9a16322235</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Mikhail Chrestkha]]></dc:creator>
            <pubDate>Thu, 04 Jan 2024 04:44:11 GMT</pubDate>
            <atom:updated>2024-01-04T05:25:01.282Z</atom:updated>
            <content:encoded><![CDATA[<p>*Title generated by Gemini Pro on Vertex AI</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qXWgmvoIpH6tVyskblbsTw.png" /><figcaption>image by author</figcaption></figure><p>I ended my last LinkedIn post of 2023 with <em>“Don’t get overwhelmed with all the new [AI] research, tools, and launches. Start building a mental model of what capabilities &amp; tools you actually need for your use cases.” </em>I took my own advice and scanned my 2023 LinkedIn posts and identified the 5 key takeaways &amp; themes I learned along the way.</p><ul><li>Which layer of the AI tech stack is right for you?</li><li>LLMs vs. LLM Systems &amp; Applications</li><li>Deploy AI applications, NOT just models</li><li>Where does your unique data come into play for GenAI?</li><li>Which type of AI models &amp; tools does your use case and application need?</li></ul><p>Full posts below for consolidated reading:</p><h3><strong>(Feb 23&#39;) Which layer of the AI tech stack is right for you?</strong></h3><p><a href="https://www.linkedin.com/posts/mchrestkha_which-layer-of-the-ai-tech-stack-is-right-activity-7030801339534237697-CFvi?utm_source=share&amp;utm_medium=member_desktop">https://www.linkedin.com/posts/mchrestkha_which-layer-of-the-ai-tech-stack-is-right-activity-7030801339534237697-CFvi</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dGFXnHET0C_vhBRtknhs0A.jpeg" /><figcaption>image by author</figcaption></figure><p>Which layer of the AI tech stack is right for you?</p><p>Gone are the days (for most of us) of needing to craft neural networks layer by layer and neuron by neuron on a scarce GPU server. The AI tools ecosystem continues to evolve at a fast pace providing higher level abstractions for ease of use and deployment. But with more choices comes more confusion and risk.</p><p>Choose too low and you may get stuck maintaining infrastructure and boilerplate code rather than innovating. Choose too high and you may get stuck with a generic solution that is hard to customize and doesn’t fully solve your use case. The perfect answer of course doesn’t exist and will often vary based on your use case, team skillset and data availability.</p><p>The diagram and list below provides an example spectrum of AI software, AI infrastructure, and AI users to consider when choosing your AI tech stack.</p><p>From bottom to top:<br>- AI researchers pushing the leading edge with lower level frameworks like PyTorch, TensorFlow, Keras Functional API, and JAX. They tend to want low level access via CLI to virtual machines and clusters to optimize and debug their code.<br>- Data Scientists &amp; ML Engineers looking to accelerate experimentation, simplify code management, and ship code faster. They tend to work with a mix of off-the-shelf OSS models (KerasNLP, HuggingFace, TFHub) and higher level frameworks like SKLearn, XGBoost, Keras Sequential API, PyTorch Lightning, &amp; Fastai. Instead of managing underlying infrastructure they prefer managed platforms to increase velocity of ideation to production deployment.<br>- Developers looking to embed AI capabilities into their applications with node.js, ruby, python, golang, etc. They tend to look for fully managed API endpoints that require a simple input and provide an AI predicted or generated response output. They tend to apply creativity to add AI-powered features into their product and applications.<br>- Business users looking for an e2e SaaS application to solve a business task or business process. They tend to look for AI assistance within a full GUI-based tool that requires no coding knowledge. Examples range from large ERP &amp; CRM systems with AI capabilities (eg. SAP, Anaplan, SalesForce Einstein) to smaller narrow applications (eg. Jasper AI, FLYR Labs, Google Docs, Google Sheets)</p><p>Which layer do you play in? I have a soft spot for the green but am amazed by what you can now do in the top 2 layers and still wowed by the innovation happening at the bottom layer (JAX, PyTorch, TPUs, NVIDIA GPUs). As we pair Predictive AI capabilities with Generative AI capabilities it will be interesting to see the impact on each of these layers.</p><h3>(Apr 23&#39;) LLMs vs. LLM Systems &amp; Applications:</h3><p><a href="https://www.linkedin.com/posts/mchrestkha_coming-back-after-3-weeks-off-felt-like-i-activity-7047090541561319424-95s8/">https://www.linkedin.com/posts/mchrestkha_coming-back-after-3-weeks-off-felt-like-i-activity-7047090541561319424-95s8/</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sSYo3NUiM46oAEiDnDx7wg.jpeg" /><figcaption>image by author</figcaption></figure><p>Coming back after 3 weeks off felt like I was away for a year in AI. AI and specifically large language models (LLMs) are moving that fast! It’s easy to feel lost. Take a breath. Choose an area that excites you, learn the fundamentals, and get prompting!</p><p>I caught up with Mr. BigQuery &amp; Data Cloud <a href="https://www.linkedin.com/in/joe-malone-84745325/">Joe Malone</a> today and we talked about how to step back and get familiar with the bigger concepts rather than get overwhelmed by every new AI research paper, model, and tool hitting the news. Below is one approach that I’m using to calm my mind!</p><p>(1) What area excites me the most? Is it the (a) AI research, (b) the tooling to better harness the tech, or © the application of this tech to a particular industry or use case?</p><p>Currently for me it’s (b), the tooling, best practices and architectures to leverage this technology most effectively. This will allow me to help users, teams and companies with ©.</p><p>(2) Learn the bigger fundamentals and concepts. After my conversation with <a href="https://www.linkedin.com/in/joe-malone-84745325/">Joe Malone</a> and a discussion with <a href="https://www.linkedin.com/in/rajeshthallam/">Rajesh Thallam</a> <a href="https://www.linkedin.com/in/mukulgoyal/">Mukul Goyal</a> &amp; <a href="https://www.linkedin.com/in/druva-reddy/">Druva Reddy Tiruvuru</a> I quickly sketched out the image below for some of the themes. I realized there were 3 areas I want to keep a pulse on:</p><p>- the LLMs themselves with their natural language interface<br>- the data, systems, and pipelines we want to augment them with<br>- the new AI tools and concepts to get familiar with including prompting (zero shot, few shot), embeddings, vector databases, tuning pipelines</p><p>(3) Nothing beats getting your hands dirty. Play and prototype with any tool you have access to for a few hours a week. Last I spoke with <a href="https://www.linkedin.com/in/thomasfrantzen/">Tom Frantzen</a> he had built a trip itinerary planner (integrated with the google maps SDK) and a wine recommender in a few days using LLMs APIs.</p><p>-Chat services: Bard, ChatGPT, You, Bing<br>-Language APIs: GPT, Cohere, Anthropic, AI21, PaLM &amp; Vertex LLMs<br>-Image APIs: MidJourney, Dalle<br>-OSS libraries: KerasCV, KerasNL, HuggingFace Transformers<br>-Any SaaS app that is built on these models: Replit, Jasper, Canva, etc.</p><p>So what am I going to try next week? I’ll be playing with LangChain (<a href="https://lnkd.in/gywDtsBB">https://lnkd.in/gywDtsBB</a>) while trying the approaches <a href="https://www.linkedin.com/in/valliappalakshmanan/">Valliappa Lakshmanan</a> laid out in his great blog <a href="https://lnkd.in/gmimhGNP">https://lnkd.in/gmimhGNP</a></p><h3>(May 23&#39;) Deploy AI applications, NOT just models</h3><p><a href="https://www.linkedin.com/posts/mchrestkha_multi-task-foundation-machine-learning-activity-7057247082230808576-t_Q6">https://www.linkedin.com/posts/mchrestkha_multi-task-foundation-machine-learning-activity-7057247082230808576-t_Q6</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*K_SwzcoEHvfm-xkKhwj1PA.jpeg" /><figcaption>image by author</figcaption></figure><p>Multi-task Foundation (Machine Learning) Models are the magic behind new predictive and generative AI capabilities. Here’s a before and after look at why developers should be excited about the potential of Foundation Models.</p><p>Even in many “AI-first” applications last year, the ML models and the AI systems were a small part of an application. They required a mix of single-task ML models and heuristics for their AI system (eg. recommendation systems, search engines, image classifiers, fraud detectors, chat bots). As a developer you put in a lot of work into an ML model that may (1) never get put into the application (2) power a small feature or (3) best case support an AI system core to the application.</p><p>Today, one multi-task foundation model can power the core AI engine behind an entire application. Chaining together natural language prompts allows you to analyze, extract, troubleshoot, explain, summarize and generate data to create novel capabilities and experiences within your application. Not only can developers accelerate how they prototype, experiment and build new AI capabilities, but they also can make a much bigger impact on the broader application, if not build it fully themselves.</p><p>The picture on the left has been discouraging to me the last few years. I just haven’t had time to even try to build any projects outside of a few models in Jupyter notebooks.</p><p>It is still very early and we have a lot to learn about foundation models (especially when it comes to safety, privacy and copyright attribution) but even as I’ve moved to a non-technical role the potential of the picture on the right has re-energized me!</p><h3>(Nov 23&#39;) Where does your unique data come into play for GenAI?</h3><p><a href="https://www.linkedin.com/posts/mchrestkha_weve-all-heard-some-form-of-your-data-is-activity-7121033737714233344-B5kc">https://www.linkedin.com/posts/mchrestkha_weve-all-heard-some-form-of-your-data-is-activity-7121033737714233344-B5kc</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RK9_ER-_OG0a_fCeICrz-A.jpeg" /><figcaption>image by a</figcaption></figure><p>We’ve all heard some form of ‘your data is your differentiator’ but in the world of off-the-shelf Generative AI models, where does your data drive the most value?</p><p>All of us from the Predictive AI era are used to the mindset that data is most important in building a train/validation/test modeling dataset early in the ML lifecycle. Once you trained the model you still had a lot to do in building an inference pipeline, but you generally knew the input data features and output format of your model endpoint.</p><p>Now you can just grab a model pre-trained by Meta, TII, OpenAI, Google, Anthropic, Cohere, Minstral AI, etc and start generating predictions. And these predictions can be large chunks of generated content! Where does my data actually add value in this new world?</p><p>With Generative AI your unique data is just as important (if not more) than Predictive AI but in different ways. Here are some thoughts and a diagram inspired by conversations with <a href="https://www.linkedin.com/in/vincentciaravino/">Vincent Ciaravino</a> <a href="https://www.linkedin.com/in/lillian-mcnealus-b3659b4b/">Lillian McNealus</a> Jennifer Otitigbe and <a href="https://www.linkedin.com/in/alm001/">Alex Martin</a>.</p><p>(1) Your unique data becomes more important during the serving step of the model lifecycle. This includes the fast evolving Retrieval Augmented Generation (RAG) design pattern where you retrieve (generally through embeddings and similarity search) and feed relevant data as input into the model to generate a response. <a href="https://www.linkedin.com/in/remy-welch-77488448/">Remy Welch</a> put together a fun example of using RAG with Vertex AI’s Text Embeddings and PaLM language model on her own book! <a href="https://lnkd.in/gRdenmVq">https://lnkd.in/gRdenmVq</a></p><p>(2) Higher quality curated data becomes very important to fine-tune these large multi-task models. Parameter Efficient Fine-Tuning (PEFT) is a fast evolving space which allows you to focus and re-direct these large models for your specific task (eg. classifying content based on your taxonomy, formatting the output for your needs). Here is a great tutorial from <a href="https://www.linkedin.com/in/nikitanamjoshi/">Nikita Namjoshi</a> on supervised tuning on Vertex AI <a href="https://lnkd.in/g28xzt9z">https://lnkd.in/g28xzt9z</a></p><p>(3) The most important data may be the data you don’t have today. This will require patiently &amp; carefully building a pipeline to collect your predictions and a way to collect preferences or corrected outputs. This is the promise of RLHF tuning which is available to use on Vertex AI Platform from <a href="https://www.linkedin.com/in/sihui-hu/">Sihui (May) Hu</a> and <a href="https://www.linkedin.com/in/yixin-wang/">Yixin (Bethany) Wang</a> <a href="https://lnkd.in/gT4zzuEH">https://lnkd.in/gT4zzuEH</a></p><p>(4) Finally, if you believe you do have lots of unique data AND the appetite for some expensive R&amp;D, you can explore using your own data to pre-train your models at the very beginning of the ML lifecycle. TPUs and NVIDIA GPUs on Vertex AI are available for those who want to take the plunge.</p><p>What are your thoughts? Feel free to leave a comment on how your AI workflows are shifting and where in the lifecycle your unique data is driving the most value.</p><h3>(Dec 23&#39;) Which type of AI models &amp; tools does your use case and application need?</h3><p><a href="https://www.linkedin.com/posts/mchrestkha_im-over-here-still-learning-about-tuning-activity-7140928955401838594-dfme">https://www.linkedin.com/posts/mchrestkha_im-over-here-still-learning-about-tuning-activity-7140928955401838594-dfme</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*idbJ4BoL5NuH9F05Ox-cZw.jpeg" /><figcaption>image by author</figcaption></figure><p>I’m over here still learning about tuning and RAG while everyone seems to have moved on to agents and multimodal?! What a time to be in the AI space!</p><p><a href="https://www.linkedin.com/in/lillian-mcnealus-b3659b4b/">Lillian McNealus</a> asked our team to get some rest tonight after a fun &amp; busy few months leading up to today’s Gemini (and more) launches but I had to end the day with a recap of my favorite announcements:</p><p>- Want to get started with Gemini in minutes with a gmail account &amp; API key for FREE? Head over to <a href="https://ai.google.dev/">https://ai.google.dev/</a> (Lauren Usui <a href="https://www.linkedin.com/in/nate-keating-02157963/">Nate Keating</a> <a href="https://www.linkedin.com/in/manvinder/">Manvinder Singh</a>)<br>- Need enterprise scale or already a Google Cloud user? Test Gemini on Vertex AI for FREE til mid-January on Vertex AI Platform <a href="https://lnkd.in/gfxCm-wp">https://lnkd.in/gfxCm-wp</a> with a brand new samples folder in our github repo <a href="https://lnkd.in/g_BKAEz7">https://lnkd.in/g_BKAEz7</a> (<a href="https://www.linkedin.com/in/keelinmcdonell/">Keelin McDonell</a> <a href="https://www.linkedin.com/in/davidlelliott/">Dave Elliott</a> <a href="https://www.linkedin.com/in/polonglin/">Polong Lin</a> <a href="https://www.linkedin.com/in/erwinhuizenga/">Erwin Huizenga</a> <a href="https://www.linkedin.com/in/nikitanamjoshi/">Nikita Namjoshi</a>)<br>- Already using <a href="https://www.linkedin.com/company/langchain/">LangChain</a> or <a href="https://www.linkedin.com/company/llamaindex/">LlamaIndex</a>, Gemini examples are ready to try [LlamaIndex: <a href="https://lnkd.in/gBPMjtxH">https://lnkd.in/gBPMjtxH</a>] [LangChain: <a href="https://lnkd.in/g7-ejKKQ">https://lnkd.in/g7-ejKKQ</a>] <a href="https://www.linkedin.com/in/kunte/">Abhi Kunté</a> <a href="https://www.linkedin.com/in/mukulgoyal/">Mukul Goyal</a> <br>- Need function calling to more confidently generate method requests from Gemini? <a href="https://www.linkedin.com/in/julia-eschelbeck/">Julia Wiesinger</a> and <a href="https://www.linkedin.com/in/goldenson/">Mark Goldenson</a> got you covered <a href="https://lnkd.in/gu-Yf-4Y">https://lnkd.in/gu-Yf-4Y</a><br>- Want to maintain large model quality at the cost and latency of a smaller model for a specialized task? <a href="https://www.linkedin.com/in/sihui-hu/">Sihui (May) Hu</a> &amp; <a href="https://www.linkedin.com/in/yixin-wang/">Yixin (Bethany) Wang</a> launched Distillation step by step as a tuning pipeline on Vertex AI <a href="https://lnkd.in/gTmxM9Er">https://lnkd.in/gTmxM9Er</a><br>- How about grounding your LLMs on your private Enterprise Data with citations without having to deal with embeddings &amp; vectorDBs? Just point to a Google Cloud Storage bucket or other GCP data sources. <a href="https://www.linkedin.com/in/alanblount/">Alan Blount</a> <a href="https://www.linkedin.com/in/diem-vu-277a49/">Diem Vu</a> and <a href="https://www.linkedin.com/in/lewisxl/">Lewis Liu</a> have an easy button for you with a great demo and blog write up from <a href="https://www.linkedin.com/in/saschaheyer/">Sascha Heyer</a> <a href="https://lnkd.in/gDq6424R">https://lnkd.in/gDq6424R</a><br>- And most importantly Vertex AI’s continued investment in safety filters &amp; scores, indemnity protection, data privacy, and enterprise security &amp; compliance <a href="https://www.linkedin.com/in/neama/">Neama Dadkhahnikoo</a> <a href="https://www.linkedin.com/in/vincentciaravino/">Vincent Ciaravino</a></p><p>Two concluding thoughts:</p><p>(1) Don’t get overwhelmed with all the new research, tools, and launches. Start building a mental model of what capabilities &amp; tools you actually need for your use cases. You don’t need Gemini to forecast inventory but Gemini can unlock new multimodal customer experiences. You probably don’t need RAG or grounding to generate creative content but you will need grounding for a chat assistant helping with instructions from company manuals.</p><p>(2) Start playing, testing and building. Even <a href="https://www.linkedin.com/in/nenshadbardoliwalla/">Nenshad Bardoliwalla</a> finds time to use Gemini to describe the interior and exterior of a rental property for a listing using video, image and text input! <a href="https://lnkd.in/g6MxFbvF">https://lnkd.in/g6MxFbvF</a> . <a href="https://www.linkedin.com/in/shriramchary/">Shri Chary</a> <a href="https://www.linkedin.com/in/joy-bonggao-27bb88122/">Joy Bonggao</a>, <a href="https://www.linkedin.com/in/dan-diasio/">Dan Diasio</a> and I have a new year’s resolution to shake the cobwebs off our hands-on-keyboard skills from our time as data tech consultants!</p><p>— — — — — — — — — — — — — — — —</p><p>Follow me on Linkedin as I post my learnings on AI going into 2024: <a href="https://www.linkedin.com/in/mchrestkha/">https://www.linkedin.com/in/mchrestkha/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee9a16322235" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Accelerated Big Data Workflow for the Data Analyst]]></title>
            <link>https://medium.com/data-science/an-accelerated-big-data-workflow-for-the-data-analyst-5921c9fda866?source=rss-9deb2fadae9e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5921c9fda866</guid>
            <category><![CDATA[rapids-ai]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[Mikhail Chrestkha]]></dc:creator>
            <pubDate>Thu, 14 Jan 2021 06:05:54 GMT</pubDate>
            <atom:updated>2021-01-14T20:07:24.526Z</atom:updated>
            <content:encoded><![CDATA[<h3>Powerful yet familiar workflow for the Data Analyst</h3><h4>Analyze billions of records faster with RAPIDS &amp; Nvidia GPUs on Google Cloud’s AI Platform Notebooks.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GeJmqw8bNWCb9zbxj06nmQ.png" /><figcaption>Photo by <a href="https://unsplash.com/@firmbee">Wiliam Iven</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><blockquote>“Of the cast of characters mentioned … the only ones every business with data needs are decision-makers and analysts”</blockquote><blockquote>— Cassie Kozyrkov (<a href="https://towardsdatascience.com/data-sciences-most-misunderstood-hero-2705da366f40">Data Science’s Most Misunderstood Hero</a>)</blockquote><p>Analysts and “<a href="https://blogs.gartner.com/carlie-idoine/2018/05/13/citizen-data-scientists-and-why-they-matter/">citizen data scientists</a>” are the often forgotten heroes across every organization . They tend to have a wide range of responsibilities spanning business domain knowledge, data extraction &amp; analysis, predictive analytics &amp; machine learning, and reporting &amp; communication to stakeholders. Piece of cake right?</p><p>And as the size of data has grown, many of these practitioners have had to learn parts of big data frameworks and infrastructure management. This increased scope of work is not sustainable and has a direct impact on the most important steps of the workflow: data exploration &amp; experimentation. This can result in rudimentary reports, less accurate predictive models &amp; forecasts, and less innovative insights &amp; ideas.</p><p>Am I really suggesting yet another big data framework/service? Don’t we already have Hive, Impala, Presto, Spark, Beam, BigQuery, Athena, and the list goes on? Don’t get me wrong. For teams running data platforms for a large organization, one or more of these frameworks/services is essential for managing hundreds of batch and streaming jobs, a vast ecosystem of data sources, and production pipelines.</p><p>My focus here however, is the data analyst who wants a flexible and scalable solution with minimal code changes to accelerate their existing workflows. Before thinking about multi-node clusters and new frameworks, you’d be surprised how much can be done with your existing code on one machine with some help from GPUs. Using myself as a guinea pig, I wanted to explore a workflow with the following constraints:</p><ol><li>I want the setup (both hardware and software) to be easy and quick (&lt; 30 min)</li><li>I don’t want to manage a distributed cluster or learn a new framework</li><li>I want full flexibility to interact with the python data &amp; machine learning ecosystem (jupyter, pandas, xgboost, tensorflow, pytorch, sklearn)</li><li>I want to be able to scale to 100s of millions of rows of data without waiting overnight for the results</li></ol><p>These constraints led me to RAPIDS with the help of the new &amp; powerful Nvidia <a href="https://cloud.google.com/blog/products/compute/announcing-google-cloud-a2-vm-family-based-on-nvidia-a100-gpu">A100 GPU</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R9mBsmkHDSmXq5EL_BqksA.png" /><figcaption>Image by author</figcaption></figure><h3>Why RAPIDS + Nvidia A100 GPUs on Google Cloud</h3><ul><li><strong>Speed</strong>: GPUs have been at the forefront of machine learning innovation by speeding up the deep learning training step in a pipeline. <a href="https://rapids.ai/">RAPIDS</a> is looking to provide similar gains across your entire workflow from data ingestion, data transformations, and predictive analytics. <a href="https://dask.org/index.html">Dask</a> extends RAPIDS to be able to scale across GPUs on a single cluster (and multi-node clusters if needed) and handle source data spread across 100s of files.</li><li><strong>Interactive Python familiarity</strong>: RAPIDS for the most part follows the pandas and sklearn API and allows for a comfortable interactive workflow to analyze results after each step. The barrier to entry is much lower than learning a new framework with different concepts. There is a reason why <a href="https://koalas.readthedocs.io/en/latest/#">Spark</a> and <a href="https://beam.apache.org/blog/dataframe-api-preview-available/">Beam</a> are both also heavily investing in the pandas API.</li><li><strong>Scale at your own pace on-demand: </strong>Only use and pay for the the Nvidia A100 GPUs when you need to. Start with Pandas on CPUs &amp; system RAM, test out RAPIDS on affordable T4 GPUs (or even for free on <a href="https://www.kaggle.com/docs/efficient-gpu-usage">Kaggle</a> &amp; <a href="https://research.google.com/colaboratory/faq.html">Colab</a>) and then scale up to Nvidia A100s for your largest datasets.</li><li><strong>Nvidia A100s on Google Cloud: 16 x 40GB GPU RAM</strong>: While you can creatively design around RAM limitations (i.e. partitions, offloading summary aggregations to a data warehouse) fitting data into GPU RAM allows for optimal speedup and full flexibility of analyzing data at the most granular level. Nvidia A100s have 40GB of GPU RAM and you can scale up to 16 of them on a single machine without having to worry about multi-node distribution. This gives you a whopping 640GB GPU RAM to play with. This both simplifies managing your environment and limits any networking issues that pop up when dealing with multi-node setups.</li></ul><p>GPUs are not a silver bullet so<a href="https://medium.com/@mchrestkha/cost-benefit-of-gpus-for-data-and-machine-learning-f7ce86e5a20f?sk=aa03a4ac438b3f279133423d5862ede2"> take a look at my previous blog</a> on assessing if GPUs are the right choice for you.</p><h3>End-to-End Example</h3><p><a href="https://github.com/mchrestkha/machine_learning_examples/blob/master/fannie_mae_loans/rapids_xgboost/notebooks/dask_rapids.ipynb">GitHub Notebook HERE</a></p><h4>Business problem</h4><p>I will be exploring the <a href="https://docs.rapids.ai/datasets/mortgage-data">Fannie Mae Single-Family Loan Performance Data</a> looking at 17 years (2000–2016) of performance history across ~37M loans and ~1.8B records. A few questions I aim to answer:</p><p>Descriptive Analytics:</p><ul><li>What are number of loans originated by year?</li><li>What % of loans had delinquency events?</li><li>What years had the highest delinquency rate?</li></ul><p>Predictive Analytics:</p><ul><li>Can we predict the likelihood of delinquency for a future loan?</li></ul><h4>Hardware Setup</h4><p>I first started off creating a managed JupyterLab environment on <a href="https://cloud.google.com/ai-platform/notebooks/docs/create-new">AI Platform Notebooks</a> using a Python 3, CUDA 11.0, and Ubuntu 18.04 image. I chose a machine with 2 Nvidia A100 GPUs but feel free to start with 1 or 2 T4 GPUs to test on a smaller sample of data. Nvidia A100s are currently only available in the us-central1 region in case you don’t see it in the dropdown.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*14L3xBoXGXvvD8JsUEqjoA.png" /><figcaption>Image by author</figcaption></figure><h4>Software Setup</h4><p>Setting up a proper environment with many moving parts (GPU Drivers, CUDA, NCCL, RAPIDS, Python, etc.) can be complex and painful. I’ve already selected Ubuntu 18.04, Python 3 and CUDA 11.0 in our base image so we need to build the right compatible RAPIDS environment. Lucky for me the <a href="https://rapids.ai/start.html">RAPIDS site</a> has a great UI-tool that generates the exact conda command based on your needs. I added <em>gcsfs</em> to access files in Google Cloud Storage and <em>matplotlib</em> to generate some charts. Setting up both the hardware and software environment took me &lt;20 min.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pIvT63or6kGr0yEFqBJcsQ.png" /><figcaption>source: <a href="https://rapids.ai/start.html">https://rapids.ai/start.html</a></figcaption></figure><pre>conda create -n rapids-0.17 -c rapidsai -c nvidia -c conda-forge -c defaults rapids-blazing=0.17 python=3.7 cudatoolkit=11.0 matplotlib=3.3.3 gcsfs=0.7.1</pre><h4>Dask Initialization</h4><p>Since my JupyterLab environment has 2 GPUs I will use dask to orchestrate RAPIDS treating each GPU as a worker. This is straightforward with 2 import statements and 2 cluster initialization lines.</p><pre>from dask_cuda import LocalCUDACluster<br>from dask.distributed import Client</pre><pre>cluster = LocalCUDACluster()<br>client = Client(cluster)</pre><h4>Data Ingestion</h4><p>You can download the full 17 years of data or a smaller subset to test this workflow from this <a href="https://docs.rapids.ai/datasets/mortgage-data">Nvidia demo repository</a>. I have 100+ csv files totaling ~200GB for the full 17 years sitting in GCS and can read them using the familiar pandas approach. You can also download the files to a local filesystem and update the location accordingly.</p><pre>csv_acq_fnames=&#39;gs://mchrestkha-github-ml-examples/fannie_mae_loans/acq/Acquisition_20*&#39;<br>csv_perf_fnames=&#39;gs://mchrestkha-github-ml-examples/fannie_mae_loans/perf/Performance_20*&#39;</pre><pre>df_acq = dask_cudf.read_csv(csv_acq_fnames, sep=&#39;|&#39;, names=col_acq_names, dtype=dtype_acq, columns=col_acq, parse_dates=parse_dates_acq)</pre><pre>df_per = dask_cudf.read_csv(csv_perf_fnames, sep=&#39;|&#39;, names=col_per_names, dtype=dtype_per, columns= col_per, parse_dates=parse_dates_per)</pre><h4>Data Exploration</h4><p>One of the main reasons to use RAPIDS+dask is to leverage all your favorite pandas functions but running faster on GPUs across multiple workers. Below are a few examples I used for initial data exploration and profiling.</p><pre>df_acq[&#39;OrYr&#39;]=df_acq[&#39;OrDate&#39;].str[-4:]<br>df_acq[&#39;OrUnpaidPrinc $M&#39;]=df_acq[&#39;OrUnpaidPrinc&#39;]/1000000</pre><pre>df_acq.head()</pre><pre>df_acq_describe=df_acq.describe().compute()</pre><pre>df_acq_nulls=df_acq.isna().sum().compute()</pre><pre>df_acq_summary = df_acq.groupby(&#39;OrYr&#39;,as_index=False).agg({&#39;LoanID&#39;: &#39;count&#39;,&#39;OrUnpaidPrinc $M&#39;: &#39;sum&#39;}).compute()</pre><p>After understanding the structure, granularity, and information in the data, I define what I’m looking to predict. Two columns are interesting when looking at the <a href="https://loanperformancedata.fanniemae.com/lppub-docs/FNMA_SF_Loan_Performance_Glossary.pdf">data dictionary</a>. The <em>‘Current Loan Delinquency Status</em>’ can be used to flag whether a loan was ever delinquent and the ‘<em>ForeclosureDate</em>’ can be used to check if a property was foreclosed. I create flags for each event and can treat either as my target variables (aka label) depending on the business question I am trying to answer. For this example I am looking to predict whether a loan will have any delinquency event so I will use the former as a label.</p><pre>df_per[&#39;DelinquentEvent&#39;]=0<br>df_per[&#39;DelinquentEvent&#39;]=df_per[&#39;DelinquentEvent&#39;].where(df_per[&#39;CLDS&#39;]&lt;1,1)</pre><pre>df_per[&#39;ForeclosureEvent&#39;]=0<br>df_per[&#39;ForeclosureEvent&#39;]=df_per[&#39;ForeclosureEvent&#39;].where(df_per[&#39;ForeclosureDate&#39;].isnull()== True,1)</pre><p>One big takeaway for me was that while there are limitations for plotting charts directly from a <em>dask_cudf</em> or <em>cudf</em> dataframes you can design around that quite easily. I used the following approach:</p><ul><li>Use GPUs + Dask for the heavy duty number crunching to create a summarized view</li><li>Convert the summarized dataframe back to pandas</li><li>Use any MatPlotlib library to plot your favorite charts using system RAM</li></ul><p>Here is an example of summarizing the full 1.8B records and then plotting a time series chart of delinquency rates by year</p><pre>df_per_yr_summary = df_per.groupby(&#39;YrRep&#39;,as_index=False).agg({&#39;LoanID&#39;: &#39;count&#39;, &#39;DelinquentEvent&#39;:&#39;sum&#39;}).compute()</pre><pre>df_per_yr_summary[&#39;DelinquencyEventsperLoan&#39;]=df_per_yr_summary[&#39;DelinquentEvent&#39;]/df_per_yr_summary[&#39;LoanID&#39;]</pre><pre>df_per_yr_summary.to_pandas().sort_values(by=[&#39;YrRep&#39;]).plot.line(x=&#39;YrRep&#39;,y=&#39;DelinquencyEventsperLoan&#39;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/384/1*YImT0c_XVAw14XNo4aUGYw.png" /><figcaption>Image by author</figcaption></figure><h4>Modeling Dataset</h4><p>I then aggregate the data to one row (aka example) per loan and join with all the loan attributes (features) that were only available at origination. This ensures I have no data leakage.</p><pre>df_per_loan = df_per.groupby(&#39;LoanID&#39;,as_index=<strong>False</strong>).agg({&#39;DelinquentEvent&#39;:&#39;sum&#39;}).compute()</pre><pre>df_per_loan[&#39;DelinquentFlag&#39;]=0<br>df_per_loan[&#39;DelinquentFlag&#39;]=df_per_loan[&#39;DelinquentFlag&#39;].where(df_per_loan[&#39;DelinquentEvent&#39;]&lt;1,1)</pre><pre>joined=df_acq.merge(df_per_loan,on=[&#39;LoanID&#39;],how=&#39;left&#39;)</pre><p>I then select my features and convert categorical variables into indicator variables (one-hot encoding).</p><pre>label=[&#39;DelinquentFlag&#39;]<br>cat_features=[&#39;Channel&#39;,&#39;OccStatus&#39;,&#39;FTHomeBuyer&#39;,&#39;LoanPurpose&#39;,&#39;PropertyType&#39;,&#39;ProductType&#39;,&#39;RelMortInd&#39;]<br>num_features=[&#39;OrInterestRate&#39;,&#39;OrUnpaidPrinc&#39;,&#39;OrLoanTerm&#39;,&#39;OrLTV&#39;,&#39;OrCLTV&#39;,&#39;CreditScore&#39;]<br>modeling_dataset=joined_categorized[cat_features + num_features + label]</pre><pre>modeling_dataset=dask_df.get_dummies(modeling_dataset)</pre><h4>Training</h4><p>I then finally set up my input parameters for XGBoost and train the model.</p><pre>&lt;define feature columns&gt;</pre><pre>X = joined[features]<br>y = joined[&#39;DelinquentFlag&#39;]<br>dtrain=xgb.dask.DaskDeviceQuantileDMatrix(client, X,y)</pre><pre>param =  {<br>               &#39;max_depth&#39;: 8,<br>               &#39;objective&#39;: &#39;reg:squarederror&#39;,<br>               &#39;tree_method&#39;: &#39;gpu_hist&#39;<br>             }<br>bst = xgb.dask.train(client,param, dtrain,num_boost_round=100)</pre><h3>Summary</h3><p>The big data ecosystem continues to evolve with a lots of choices (Spark, Beam, Flink, Dask, Modin, Ray, BigQuery, SnowFlake, Athena, etc.). Historically many of these options required a tradeoff between usability, flexibility and scale. Luckily many of these projects are investing to reduce the need for these tradeoffs.</p><p>I am most excited on the impact this will have on the unsung heroes of the data world, data analysts. Empowering data analysts and citizen data scientists to explore &amp; analyze big data seamlessly has tremendous benefits to a business: more insightful BI reporting, more accurate forecasts &amp; predictive models, and more data-driven &amp; innovative decision-making.</p><p>I offer up one workflow with RAPIDS on Google Cloud’s AI Platform Notebooks with GPUs that any data analyst can leverage today. This architecture follows the principles below that make it a great first step to start accelerating your big data (and data science) workflows before potentially expanding to Spark or Beam for more complex pipeline orchestration needs.</p><ul><li>Quick and easy setup (&lt; 30 min)</li><li>No distributed cluster; Scale vertically on one machine before adding horizontal scaling complexity</li><li>Familiar python tools and APIs (pandas for the win!)</li><li>Scale to 100s of millions of rows of data</li><li>Accelerate the entire data life cycle</li></ul><p>Big data and machine learning technologies are evolving as fast as ever. High tech companies and research departments will continue to push the limits for their advanced needs. It is however important to step back and think about what is available today. I believe there is a HUGE backlog of use cases that data analysts can solve better &amp; faster with the tools available today. I am excited for many projects (RAPIDS, Dask, BQML, Koalas on Spark) focusing on scalability AND usability to lower the barrier to entry and empower a much larger part of the data community.</p><p>Have a question or want to chat? Find me on <a href="https://twitter.com/mchrestkha">Twitter</a></p><p>The notebook examples from this blog can be found on <a href="https://github.com/mchrestkha/machine_learning_examples/blob/master/fannie_mae_loans/rapids_xgboost/notebooks/dask_rapids.ipynb">my GitHub</a>.</p><p>Big thanks to <a href="https://www.linkedin.com/in/ethem-f-can-5020b556/">Ethem Can</a>, <a href="https://www.linkedin.com/in/remy-welch-77488448/">Remy Welch</a>, <a href="https://www.linkedin.com/in/dong-meng/">Dong Meng</a>, <a href="https://www.linkedin.com/in/arunsraman/">Arun Raman</a>, <a href="https://www.linkedin.com/in/rajeshthallam/">Rajesh Thallam</a>, <a href="https://www.linkedin.com/in/mike-thomas-a0173318/">Michael Thomas</a>, <a href="https://www.linkedin.com/in/rajan-arora-ph-d-54b6766/">Rajan Arora</a> and <a href="https://www.linkedin.com/in/subhanali/">Subhan Ali</a> for educating me and helping with the example workflow.</p><h3>FAQ</h3><ul><li><strong>[Q] </strong>Isn’t a data warehouse (like Snowflake or BigQuery) more efficient for the upstream data processing?<strong> [A]</strong> If your company/team has access to a scalable data warehouse then yes, push as much compute (that can be written with SQL) to it as you can early in your workflow. This blog’s focus was a self-contained workflow that only required object storage (or local file system) and an on-demand Jupyter notebook instance with a GPU. It follows more of a data lake architecture.</li><li><strong>[Q] </strong>Aren’t GPUs super expensive? <strong>[A] </strong><a href="https://medium.com/@mchrestkha/cost-benefit-of-gpus-for-data-and-machine-learning-f7ce86e5a20f">First read this</a> as GPUs may not be the right fit for your workflow. GPUs are however becoming the norm for the training step of machine learning models. If you plan to use GPUs for one step of your pipeline, RAPIDS allows you to run and speed up the rest of the pipeline, potentially increasing the ROI of the GPU.</li><li><strong>[Q]</strong> How does RAPIDS fit in with Spark and Beam?<strong> [A] </strong>This blog focused on a single-node setup where various RAPIDS libraries (i.e. cudf, cuml) can be used stand-alone similar to pandas. For multi-node setups, you can think of RAPIDS as a lower level GPU acceleration layer that can power different distributed big data frameworks. Spark and Dask both leverage RAPIDS under the covers when running data tasks on GPUs. I assume the Apache Beam project may also look into RAPIDS integration for GPU acceleration in the future.</li><li><strong>[Q]</strong> This doesn’t seem that fast! How much faster can this be optimized? <strong>[A] </strong>I am no expert so I’m sure there are several ways this could be sped up (i.e. optimal file sizes &amp; partitions, indexing, leveraging some unused CPU cores, etc.). This will require some trial an error. One part I plan to explore more in the future is to use the <a href="https://docs.dask.org/en/latest/diagnostics-distributed.html#dashboard">Dask Dashboard</a> to better understand bottlenecks and <a href="https://github.com/rapidsai/jupyterlab-nvdashboard">NVDashboard</a> to better understand GPU utilization. If you have a use case, dataset and benchmark, I would love to connect and help you test it.</li><li><strong>[Q]</strong> Can multiple users share the same environment to share infrastructure and GPUs? <strong>[A] </strong>Good catch! This workflow kept it simple with one user per machine (which can be shut off on-demand) but is still not cost effective for a team, especially for the more expensive Nvidia A100 or V100 GPUs. There are solutions to set up multi-tenant notebook environments on Kubernetes which do add some complexity. The Nvidia A100 is also intriguing with it’s <a href="https://www.nvidia.com/en-us/technologies/multi-instance-gpu/">multi-instance GPU (MIG) </a>architecture which allows you to slice up each GPU into 7 units so different users can run tasks on each concurrently. Something to look into for a future blog.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5921c9fda866" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/an-accelerated-big-data-workflow-for-the-data-analyst-5921c9fda866">An Accelerated Big Data Workflow for the Data Analyst</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightweight yet scalable TensorFlow workflow on Google Cloud]]></title>
            <link>https://medium.com/data-science/lightweight-yet-scalable-tensorflow-workflow-on-google-cloud-b32e0ba410c6?source=rss-9deb2fadae9e------2</link>
            <guid isPermaLink="false">https://medium.com/p/b32e0ba410c6</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[editors-pick]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Mikhail Chrestkha]]></dc:creator>
            <pubDate>Mon, 05 Oct 2020 06:07:50 GMT</pubDate>
            <atom:updated>2020-10-07T20:10:11.138Z</atom:updated>
            <content:encoded><![CDATA[<h4>My superpower toolkit: TFRecorder, TensorFlow Cloud, AI Platform Predictions and Weights &amp; Biases</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Qzkf0gVtKIBDakB8DIhQg.png" /><figcaption>Image by author using <a href="https://www.figma.com/">figma.com</a></figcaption></figure><p>I am just going to say it. I am absolutely overwhelmed and intimidated with the growing breadth and depth of machine learning (ML) today.</p><ul><li>Need to build a high performant data pipeline? Learn Apache Beam or Spark and protocol buffers</li><li>Need to scale your model training? Learn AllReduce and multi-node distributed architectures</li><li>Need to deploy your models? Learn Kubernetes, TFServing, quantization, and API management</li><li>Need to track pipelines? Set up a metadata database, learn docker, and become a DevOps engineer</li></ul><p>This does not even include the algorithm and modeling space which also makes me feel like an imposter without a research background. There has got to be an easier way!</p><p>I spent the last few weeks thinking about this dilemma and what I would recommend to a data scientist with a similar mindset as me. Many of those topics above are important to learn, especially if you want to focus on the new field of MLOps, but are there tools and technologies that can allow you to stand on the shoulder of giants?</p><p>Below are 4 such tools that abstract away much of the complexity and can allow you to more efficiently develop, track and scale your ML workflows.</p><ul><li><a href="https://github.com/google/tensorflow-recorder"><strong>TFRecorder</strong></a><strong> (via Dataflow)</strong>: Turn data into TFRecords with ease by feeding in a CSV file. For images just provide the JPEG URIs and labels in the CSV. Scale to distributed servers with Dataflow without writing any Apache Beam code.</li><li><a href="https://github.com/tensorflow/cloud"><strong>TensorFlow Cloud </strong></a><strong>(via AI Platform Training): </strong>Scale your TensorFlow model training to single and multi-node clusters of GPUs on AI Platform Training with a simple API call.</li><li><strong>AI Platform Predictions: </strong>Deploy your model as an API endpoint to a Kubernetes backed autoscaling service with GPUs, the same one used by <a href="https://cloud.google.com/blog/products/ai-machine-learning/how-waze-predicts-carpools-using-google-cloud-ai-platform">Waze</a>!</li><li><strong>Weights &amp; Biases:</strong> Log artifacts (datasets and models) to track versions and lineage across your development pipeline. Automatically generates a <a href="https://www.wandb.com/articles/announcing-artifacts">tree of relationships between your experiments and artifacts</a>.</li></ul><h3>Workflow Overview</h3><p>I will use a typical cats vs. dogs computer vision problem to walk through each of these tools. This workflow has the following steps:</p><ul><li>Save the raw JPEG images to object storage with each image located under a subfolder specifying it’s label</li><li>Generate the CSV file with image URIs and labels in the required format</li><li>Convert images and labels to TFRecords</li><li>Create a dataset from the TFRecords and train a CNN model with Keras</li><li>Store the model as a SavedModel and deploy as an API endpoint</li><li>JPEG images, TFRecords, and SavedModels will all be stored in object storage</li><li>Experiments and lineage of artifacts will be tracked with Weights &amp; Biases</li></ul><p>The notebooks and scripts I used are in this <a href="https://github.com/mchrestkha/machine_learning_examples/tree/master/catsdogs/tensorflow">GitHub repository</a>.</p><p>Now let’s dive into each tool.</p><h3>TFRecorder</h3><p>TFRecords still confuse me. I understand the performance advantage they provide but have always struggled with working with them once I start on a new dataset. Apparently I am not the only one and thankfully the TFRecorder project was recently released. It has never been easier working with TFRecords and it only requires (1) organizing your images in a logical directory format and (2) working with PANDAS DataFrames and CSVs. Below are the steps I took:</p><ul><li>Create a CSV file with 3 columns including an image URI pointing to the directory location of each image</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MybwAqvOZI3PzdvQflRSxg.png" /><figcaption>Image by author</figcaption></figure><ul><li>Read the CSV into a PANDAS DataFrame and call the TFRecorder function to convert the files on Dataflow, specifying an output directory</li></ul><pre>dfgcs = pd.read_csv(FILENAME)</pre><pre>dfgcs.tensorflow.to_tfr(<br>    output_dir=TFRECORD_OUTPUT,<br>    runner=&#39;DataFlowRunner&#39;,<br>    project=PROJECT,<br>    region=REGION,<br>    tfrecorder_wheel=TFRECORDER_WHEEL)</pre><p>Thats it! Less than 10 lines of code that can scale to converting millions of images into the TFRecord format. As a data scientist you just laid the foundation for a high performant training pipeline. You can also take a look at the Dataflow job graph and metrics in the Dataflow console if you are curious about the magic happening in the background.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iiRoS9ZdTHvq6lpVBA2zgw.png" /><figcaption>Image by author</figcaption></figure><p>After reading through a bit of the GitHub repo, the schema of the tfrecord is below:</p><pre>tfr_format = {<br>            &quot;image&quot;: tf.io.FixedLenFeature([], tf.string),<br>            &quot;image_channels&quot;: tf.io.FixedLenFeature([], tf.int64),<br>            &quot;image_height&quot;: tf.io.FixedLenFeature([], tf.int64),<br>            &quot;image_name&quot;: tf.io.FixedLenFeature([], tf.string),<br>            &quot;image_width&quot;: tf.io.FixedLenFeature([], tf.int64),<br>            &quot;label&quot;: tf.io.FixedLenFeature([], tf.int64),<br>            &quot;split&quot;: tf.io.FixedLenFeature([], tf.string),<br>        }</pre><p>You can then read the TFRecords into a TFRecordDataset for a Keras model training pipeline with the code below:</p><pre>IMAGE_SIZE=[150,150]<br>BATCH_SIZE = 5</pre><pre>def read_tfrecord(example):<br>    image_features= tf.io.parse_single_example(example, tfr_format)<br>    image_channels=image_features[&#39;image_channels&#39;]<br>    image_width=image_features[&#39;image_width&#39;]<br>    image_height=image_features[&#39;image_height&#39;]<br>    label=image_features[&#39;label&#39;]<br>    image_b64_bytes=image_features[&#39;image&#39;]<br>    <br>    image_decoded=tf.io.decode_base64(image_b64_bytes)<br>    image_raw = tf.io.decode_raw(image_decoded, out_type=tf.uint8)<br>    image = tf.reshape(image_raw, tf.stack([image_height,    image_width, image_channels]))<br>    image_resized = tf.cast(tf.image.resize(image, size=[*IMAGE_SIZE]),tf.uint8)<br>    return image_resized, label</pre><pre>def get_dataset(filenames):<br>    dataset = tf.data.TFRecordDataset(filenames=filenames, compression_type=&#39;GZIP&#39;) <br>    dataset = dataset.map(read_tfrecord)<br>    dataset = dataset.shuffle(2048)<br>    dataset = dataset.batch(BATCH_SIZE)<br>    return dataset</pre><pre>train_dataset = get_dataset(TRAINING_FILENAMES)<br>valid_dataset = get_dataset(VALID_FILENAMES)</pre><h3>TensorFlow Cloud (AI Platform Training)</h3><p>Now that we have a tf.data.Dataset, we can feed it into our model training call. Below is a simple example of a CNN model using the Keras Sequential API.</p><pre>model = tf.keras.models.Sequential([<br>    tf.keras.layers.Conv2D(16, (3,3), activation=&#39;relu&#39;, input_shape=(150, 150, 3)),<br>    tf.keras.layers.MaxPooling2D(2, 2),<br>    tf.keras.layers.Conv2D(32, (3,3), activation=&#39;relu&#39;),<br>    tf.keras.layers.MaxPooling2D(2,2),<br>    tf.keras.layers.Conv2D(64, (3,3), activation=&#39;relu&#39;),<br>    tf.keras.layers.MaxPooling2D(2,2),<br>    tf.keras.layers.Conv2D(64, (3,3), activation=&#39;relu&#39;),<br>    tf.keras.layers.MaxPooling2D(2,2),<br>    tf.keras.layers.Flatten(),<br>    tf.keras.layers.Dense(256, activation=&#39;relu&#39;),<br>    tf.keras.layers.Dense(1, activation=&#39;sigmoid&#39;)<br>])</pre><pre>model.summary()<br>model.compile(loss=&#39;binary_crossentropy&#39;,<br>              optimizer=RMSprop(lr=1e-4),<br>              metrics=[&#39;accuracy&#39;])<br>model.fit(<br>    train_dataset,<br>    epochs=10,<br>    validation_data=valid_dataset,<br>    verbose=2<br>)</pre><p>I first ran this on my development environment on a subset of the images (in my case a Jupyter notebook) but I wanted to make it scale to all images and make it faster. TensorFlow Cloud allows me to use a single API command that takes care of containerizing my code and submitting to run as a distributed GPU job.</p><pre>import tensorflow_cloud as tfc</pre><pre>tfc.run(entry_point=&#39;model_training.ipynb&#39;,<br>        chief_config=tfc.COMMON_MACHINE_CONFIGS[&#39;T4_4X&#39;],<br>        requirements_txt=&#39;requirements.txt&#39;)</pre><p>This is no April fool’s joke. The code above (&lt;5 lines of code) is the full python script you need to place in the same directory as your Jupyter notebook. The trickiest part is to follow all the set up instructions to make sure you are correctly authenticated to your Google Cloud Platform project. This is a massive super power!</p><p>Let’s dive a bit deeper into what this is doing.</p><p>First a docker container with all your required libraries and notebook will be built and saved in Google Cloud’s Container Registry service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2DHHthHcKBI0BfJbVg_ugQ.png" /><figcaption>Image by author</figcaption></figure><p>That container is then submitted to a fully managed and serverless training service, AI Platform Training. Without having to set up any infrastructure and install any GPU libraries I was able to train this model on a 16 vCPU 60GB RAM machine with 4 Nvidia T4 GPUs. I only used those resources for the time I needed them (~15 minutes) and can go back to developing in my local environment with an IDE or Jupyter Notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FZS60P978u0dcLow0JX3LA.png" /><figcaption>Image by author</figcaption></figure><p>The SavedModel is finally stored in object storage as specified at the very end of my training script.</p><pre>MODEL_PATH=time.strftime(&quot;gs://mchrestkha-demo-env-ml-examples/catsdogs/models/model_%Y%m%d_%H%M%S&quot;)<br>model.save(MODEL_PATH)</pre><h3>AI Platform Predictions</h3><p>With my SavedModel in object storage, I can load it up to my development environment and run some sample predictions. But what if I want to allow others to use it without needing them to set up a Python environment and learn TensorFlow. This is where AI Platform Predictions comes into the picture. It allows you to deploy model binaries as API endpoints that can be called using REST, a simple Google Cloud SDK (gcloud), or various other client libraries. All end users need to know is the required input (in our case a JPEG image file converted to a [150,150,3] JSON array) and can embed your model as part of their workflow. When you make a change (retraining on a new dataset, a new model architecture, maybe even a new framework) you can publish a new version.</p><p>The simple gcloud SDK below is the super power to deploy your models to this Kubernetes backed autoscaling service.</p><pre>MODEL_VERSION=&quot;v1&quot;<br>MODEL_NAME=&quot;cats_dogs_classifier&quot;<br>REGION=&quot;us-central1&quot;</pre><pre>gcloud ai-platform models create $MODEL_NAME \<br>    --regions $REGION</pre><pre>gcloud ai-platform versions create $MODEL_VERSION \<br>  --model $MODEL_NAME \<br>  --runtime-version 2.2 \<br>  --python-version 3.7 \<br>  --framework tensorflow \<br>  --origin $MODEL_PATH</pre><p><a href="https://cloud.google.com/blog/products/ai-machine-learning/ai-platform-prediction-better-reliability-ml-workflows">AI Platform Predictions</a> is a service I am particularly excited about as it eliminates many of the complexities to get your model out into the world (both internally and externally) to start driving value from it. While the scope of this blog is an experimentation workflow <a href="https://cloud.google.com/blog/products/ai-machine-learning/how-waze-predicts-carpools-using-google-cloud-ai-platform">companies like Waze</a> are using AI Platform Predictions to deploy and serve their models in production at scale.</p><h3>Weights &amp; Biases</h3><p>At this point I’ve completed one experiment but what about my future experiments? I may need to:</p><ul><li>run many more experiments later this week and track my work</li><li>come back in a month and try to remember all the inputs and outputs of each of my experiments</li><li>share the work with teammates who can hopefully piece my workflow together</li></ul><p>There is lots of work being done in the ML Pipelines space. It is an exciting but nascent space with best practices and industry standards yet to develop. Some great projects to follow include MLFlow, Kubeflow Pipelines, TFX, and Comet.ML. For the needs of my workflow, MLOps and continuous delivery was out of scope and I wanted something simple. I chose Weights &amp; Biases (WandB) due to its ease of use and lightweight integration to track experiments and artifacts.</p><p>Let’s start with experiments. WandB offers lots of customization options but if you’re using any one of the popular frameworks you don’t need to do much. For the TensorFlow Keras API I simply (1) imported the <em>wandb</em> python library (2) initialized my experiment run and (3) added a callback function within the model fit step.</p><pre>model.fit(<br>    train_dataset,<br>    epochs=10,<br>    validation_data=valid_dataset,<br>    verbose=2,<br>   <strong> callbacks=[WandbCallback()</strong>]<br>)</pre><p>This automatically streams out-of-the-box metrics into a centralized experiment tracking service. Take a look at all the ways folks are using <a href="https://www.wandb.com/articles">WandB here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*23CX5IM1NPwHt3z5KUZvrg.png" /><figcaption>Image by author</figcaption></figure><p>WandB also provides an artifacts API which fits my needs much more than some of the heavy tooling out there today. I added short code snippets throughout my pipeline to define 4 key items:</p><ol><li>Initializing a step in my pipeline</li><li>Using an existing artifact (if available) as part of this step</li><li>Logging an artifact generated by this step</li><li>Stating a step is complete</li></ol><pre>run = wandb.init(project=&#39;cats-dogs-keras&#39;, job_type=&#39;data&#39;, name=&#39;01_set_up_data&#39;)</pre><pre><strong><em>&lt;Code to set up initial JPEGs in the appropriate directory structure&gt;</em></strong></pre><pre>artifact = wandb.Artifact(name=&#39;training_images&#39;,job_type=&#39;data&#39;, type=&#39;dataset&#39;)<br>artifact.add_reference(&#39;gs://mchrestkha-demo-env-ml-examples/catsdogs/train/&#39;)<br>run.<strong>log_artifact</strong>(artifact)<br>run.finish()</pre><pre>run = wandb.init(project=&#39;cats-dogs-keras&#39;,job_type=&#39;data&#39;, name=&#39;02_generate_tfrecords&#39;)</pre><pre>artifact = run.<strong>use_artifact</strong>(&#39;training_images:latest&#39;)</pre><pre><strong><em>&lt;TFRecorder Code&gt;</em></strong></pre><pre>artifact = wandb.Artifact(name=&#39;tfrecords&#39;, type=&#39;dataset&#39;)<br>artifact.add_reference(&#39;gs://mchrestkha-demo-env-ml-examples/catsdogs/tfrecords/&#39;)<br>run.<strong>log_artifact</strong>(artifact)<br>run.finish()</pre><pre>run = wandb.init(project=&#39;cats-dogs-keras&#39;,job_type=&#39;train&#39;, name=&#39;03_train&#39;)<br>artifact = run.<strong>use_artifact</strong>(&#39;tfrecords:latest&#39;)</pre><pre><strong><em>&lt;TensorFlow Cloud Code&gt;</em></strong></pre><pre>artifact = wandb.Artifact(name=&#39;model&#39;, type=&#39;model&#39;)<br>artifact.add_reference(MODEL_PATH)<br>run.<strong>log_artifact</strong>(artifact)</pre><p>This simple Artifacts API stores the metadata and lineage of each run and artifact so you have full clarity on your workflow. The UI also has a nice tree diagram to view it graphically.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b_zPJKo7bjkiui-fITnGOQ.png" /><figcaption>Image by author</figcaption></figure><h3>Summary</h3><p>If you feel intimidated by the endless machine learning topics, tools and technologies, you are not alone. I feel imposter syndrome everyday talking to co-workers, partners and customers about various aspects of data science, MLOps, hardware accelerators, data pipelines, AutoML, etc. Just remember it is important to:</p><ol><li><strong>Be practical</strong>: It is not realistic for all of us to be full stack engineers across DevOps, Data, and Machine Learning. Pick 1 or 2 areas that interest you and work with others to solve system-wide problems.</li><li><strong>Focus on your problem</strong>: We all get carried away with the new framework, the new research paper, the new tool. Start with your business problem, your dataset, your end-user requirements. Not everyone needs 100s of ML models in production being re-trained daily and served to millions of users (at least not yet).</li><li><strong>Identify superpower tools:</strong> Find a core set of tools that act as efficiency multipliers that provide scalability and remove complexity. I walked through my tool kit for Tensorflow (<strong>TFRecorder + TensorFlow Cloud + AI Platform Predictions + Weights &amp; Biases</strong>), but find the right toolkit that maps to your problem and workflow.</li></ol><p>Have a question or want to chat? Find me on <a href="https://twitter.com/mchrestkha">Twitter</a></p><p>The notebook examples from this blog can be found on <a href="https://github.com/mchrestkha/machine_learning_examples/tree/master/catsdogs/tensorflow">my GitHub</a>.</p><p>Big thanks to <a href="https://www.linkedin.com/in/mike-bernico/">Mike Bernico</a>, <a href="https://www.linkedin.com/in/rajeshthallam/">Rajesh Thallam</a> and <a href="https://www.linkedin.com/in/1vaibhavsingh/">Vaibhav Singh</a> for helping me with this example solution.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b32e0ba410c6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/lightweight-yet-scalable-tensorflow-workflow-on-google-cloud-b32e0ba410c6">Lightweight yet scalable TensorFlow workflow on Google Cloud</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cost-Benefit of GPUs for Data and Machine Learning]]></title>
            <link>https://mchrestkha.medium.com/cost-benefit-of-gpus-for-data-and-machine-learning-f7ce86e5a20f?source=rss-9deb2fadae9e------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7ce86e5a20f</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[Mikhail Chrestkha]]></dc:creator>
            <pubDate>Mon, 24 Aug 2020 15:24:24 GMT</pubDate>
            <atom:updated>2020-08-24T15:56:56.888Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Framework to assess GPUs for your team</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xuQVNyfDjcHzbatOrjtyfQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@casparrubin">Caspar Camille Rubin</a> on <a href="https://unsplash.com/">Unsplash</a></figcaption></figure><p>When I heard the term <em>GPUs</em> as early as 4 years ago my mind immediately turned to computer graphics and video games. Little did I know that GPUs had found their way into other fields including data and machine learning. GPUs started allowing machine learning (ML) researchers to run experiments in minutes which took hours or days before. They have enabled various breakthroughs by allowing users to solve problems that required large amounts of computing which were not available or feasible prior.</p><p>Still, GPUs are not (yet) magical and general purpose. Many of us struggle to understand, use, and build a business case for GPUs. Over the last few years, I have been lucky to work with (and learn from) <a href="https://medium.com/u/ab69c39a85e1">NVIDIA AI</a> and various <a href="https://medium.com/u/4f3f4ee0f977">Google Cloud</a> customers who are leading the charge in GPU adoption and value generation. When assessing the cost-benefit of GPUs for data and machine learning you can bucket various factors into 3 groups.</p><ol><li><strong>Fit &amp; Learning Curve: </strong>Are GPUs a good fit for your task? What steps in your workflow can they help with? How much skill, setup, and potential redesign investment is required to get GPUs to work? What software is available to make GPUs easier to leverage?</li><li><strong>Cost: </strong>What type of GPUs fulfill your needs? How long does your task take to run (with and without GPUs)? What is the price per unit of time? What is the overall cost per training experiment or prediction?</li><li><strong>Value: </strong>Most organizations and teams stop at (1) and (2) and don’t consider various value elements GPUs can influence. But GPUs are much more than an IT cost line item. It is important to understand both direct and indirect value generated from adding GPUs into your system processes and architecture. These can include employee productivity gains, improved end user experience, accelerated time to market and direct revenue impact.</li></ol><p>I dive into a bit more detail on each topic below.</p><h3>Fit &amp; Learning Curve</h3><p>The first step of course is to understand if GPUs are the right fit for your team and tasks. Below are a few considerations:</p><ul><li><strong>Data Volume</strong>: The primary goal of GPUs (in our data science world) is to accelerate data and machine learning tasks. If your datasets are still small and take seconds and minutes to complete, GPUs may not be as impactful. Even so, you can build a data and ML strategy (i.e. save and acquire more data, build a data lake, pilot data science &amp; ML projects) which can plan to make use of GPUs a few quarters and years out.</li><li><strong>Tasks</strong>: What is the bottleneck of your data pipeline? Are your teams exploring large deep learning models or simpler linear models? Is your end goal to derive insights from data or ship machine learning features into production as part of your application? GPUs are a no brainer for deep learning on big data(both training and serving) but are slowly evolving to accelerate traditional ML, SQL and data processing tasks as well. It is important to map your tasks to what GPUs are good at and set realistic expectations. The slowest step of your data pipeline is not always something GPUs can assist with and may prevent you from unlocking the full potential of GPUs later in your pipeline.</li><li><strong>Skills &amp; Usability</strong>: Even if you have big data and identified the right tasks to leverage GPUs the biggest barrier may be the the skills required. GPUs require brand new skills for lower-level programming (i.e. CUDA) or extended knowledge to effectively use them with existing tools. Even if your code runs on GPUs it is most likely not efficient and optimal on the first try. Understanding how to optimize GPU utilization for your tasks is a non-trivial job. I am however most optimistic about this as GPU usability continues to improve and the breadth of tasks GPUs support continues to grow. This section alone could be it’s own blog so I’ll highlight just a few of the reasons to be bullish on GPU accessibility: (1) Nvidia’s investment in a growing <a href="https://developer.nvidia.com/">GPU developer tools</a> ecosystem including CUDA, <a href="https://rapids.ai/">RAPIDS</a>, <a href="https://developer.nvidia.com/tensorrt">TensorRT</a>, and <a href="https://ngc.nvidia.com/catalog/all">NGC</a> (2) increasing list of projects supporting GPUs natively including Spark 3.0, TensorFlow, PyTorch, XGBoost and Kubeflow.</li></ul><p>Okay, at this point you have the right data, tasks and skills for GPUs. But aren’t GPUs expensive? Let’s take a look at cost.</p><h3>Cost</h3><p>To begin with, yes, GPUs are more expensive than CPUs. That is expected for something that provides more cores, higher throughput, higher degrees of parallel processing and concurrent operations. The key is to understand the cost once you normalize for factors such as task performance and time. In addition, cloud allows you to try various types of GPUs and only pay for the time you are using them. You may use CPUs 24/7 but GPUs for a few hours a day. GPUs no longer require capital investment in hardware for physical servers (or expensive research workstations).</p><ul><li><strong>Types of GPUs</strong>: Depending on your data volume, model size, task, and system architecture there are various GPUs in the market to meet your needs across various price points. For training large computer vision or language models Nvidia’s V100s and <a href="https://cloud.google.com/blog/products/compute/announcing-google-cloud-a2-vm-family-based-on-nvidia-a100-gpu">new A100s</a> will give you the horse power but also cost the most. If you are looking to serve these models in production as part of your application P4s and T4s will do the trick. If you want to leverage GPUs on Jupyter notebooks for various data and algorithms tasks start with K80s and T4s which are the lowest cost option on most clouds today. Check out all the <a href="https://cloud.google.com/compute/gpus-pricing">Nvidia GPUs available on Google Cloud HERE</a>.</li><li><strong>CPU &amp; RAM requirements</strong>: Even with GPUs you need a server with CPUs and RAM. You are not fully replacing CPUs with GPUs but simply replacing a subset of tasks CPUs would need to perform. The key is understanding how much you can reduce your CPU and RAM needs by adding GPUs to the server. This will require trial and error and some knowledge of measuring CPU/GPU utilization and RAM usage. <a href="https://cloud.google.com/blog/products/data-analytics/ml-with-xgboost-gets-faster-with-dataproc-on-gpus">In this example from Nvidia</a>, they were able to reduce vCPU cores and RAM by half (64vCPU 416GB RAM → 32vCPU &amp; 208GB RAM) by adding 4 V100 GPUs to each node in the cluster.</li><li><strong>Price-Performance</strong>: These above choices with some experimentation allow you to create a normalized price-performance comparison per experiment for training or per prediction for serving. In the training scenario, GPUs may cost $5 per minute vs. $1 for CPUs but run an experiment 10 times faster resulting in a 2x cost reduction. It is good practice to be as specific as possible when defining an experiment for comparison (i.e. time, translated to cost, for a training job to reach 80% accuracy).</li></ul><p>If GPUs are cheaper and faster per experiment (or per prediction) your business case to start using GPUs has written itself!</p><h3>Value</h3><p>But what if in the example prior ($5 vs. $1 per minute) GPUs ran 3 times faster (not 10). Cost per experiment actually increases ~1.7x but performance is 3x faster. Is the 3x speed worth the 1.7x cost? That will be up to you to decide and leads to our 3rd, and often ignored, consideration: Value generated. Below are some ways to think about and ideally quantify value driven by GPUs.</p><ul><li><strong>Productivity Gains</strong>: How many more experiments can your practitioners run in a day/week/month? How much are you paying your practitioners to sit and wait for their algorithms and jobs to complete? If you have dedicated teams of machine learning researchers and data scientists your tolerance for slow experiments is much lower than a company with a few data analysts running weekly batch jobs. Every company’s tolerance will lie within this spectrum but the key is to be aware of the opportunity cost of not being able to run many more experiments per hour/day/month/quarter/year.</li><li><strong>Accelerated Time to Market: </strong>The end goal of faster experiments includes launching a product or feature faster to market. Allowing your organization to run experiments faster creates a compounding effect of trying new ideas, failing fast, and reaching a breakthrough outcome. This requires thinking about the broader system and capturing data that you may not have today (i.e. avg. time from idea to launch, frequency of ML model refresh, # of ML models in production).</li><li><strong>Improved End User Experience</strong>: How much faster can you serve predictions to your end-users in your application? How much does an extra 100ms of latency mean to your customers? You may have developed a more accurate model but your CPU-based serving infrastructure can only serve it at 500ms of latency due to its larger size. The gains from the improved predictive power may be offset by the impacted latency. <a href="https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-latency-cost-them-1-in-sales/">Various studies</a> have shown how increased latency can result in lost traffic and sales. GPUs provide you the ability to deploy large, complex and more accurate models without sacrificing latency hits.</li><li><strong>Direct revenue impact</strong>: Is there a direct impact on conversion? churn? engagement? This is of course the holy grail. Something every leader asks their team. What is the top line impact?! GPUs may cost more during the experimentation phase but the improved models developed can return much more value once activated in production.</li></ul><h3>Conclusion</h3><ul><li>The barrier to entry for learning and using GPUs continues to get lower. Your favorite data and ML tools today may already support GPUs.</li><li>While you should always be skeptical of the GPU benchmarks you see online (50x faster!?!), with the advent of cloud you have no reason not to try.</li><li>Even if GPUs are costlier don’t forget to evaluate the value they may return tenfold.</li></ul><p>Use the framework discussed above to assess the cost-benefit of GPUs on your own data and experiments. You may be surprised with the results.</p><p>Have a question or want to chat? Find me on <a href="https://twitter.com/mchrestkha">Twitter</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7ce86e5a20f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CatBoost on Google Cloud’s AI Platform w/ CPUs & GPUs]]></title>
            <link>https://mchrestkha.medium.com/catboost-on-google-clouds-ai-platform-w-cpus-gpus-130e4407e132?source=rss-9deb2fadae9e------2</link>
            <guid isPermaLink="false">https://medium.com/p/130e4407e132</guid>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[catboost]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Mikhail Chrestkha]]></dc:creator>
            <pubDate>Tue, 26 May 2020 04:39:12 GMT</pubDate>
            <atom:updated>2020-05-26T04:45:26.418Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Q8wxF7ppIAezaBXVSnx9w.png" /></figure><h3>Summary</h3><ul><li>CatBoost, an OSS gradient boosting framework , is the new kid on the block with intriguing benchmark results on model quality and training/serving speeds with CPUs &amp; GPUs</li><li>Catboost is simple to run on Google Cloud’s AI Platform for both the Notebook &amp; Training services. <a href="https://github.com/mchrestkha/machine_learning_examples/tree/master/census/catboost/gcp_ai_platform"><strong>GitHub examples to accompany this post HERE</strong></a></li><li>While AI Platform Training only lists TensorFlow, XGBoost, &amp; SKLearn as officially supported hosted frameworks, CatBoost works seamlessly by including it in the setup.py file during training job submission</li></ul><h3>CatBoost for Gradient Boosting</h3><p>My focus the last few years has been heavily skewed towards deep learning &amp; neural networks but this weekend I wanted to quickly catch up on the world of gradient boosting on structured data. The biggest item I seem to have missed over the last 2.5 years is the launch and growth of CatBoost<strong>.</strong></p><p>You can head over to <a href="https://catboost.ai/">catboost.ai</a> and watch this <a href="https://youtu.be/usdEWSDisS0">great intro video</a> to learn more but here are some of the highlights:</p><ul><li>OSS ML framework from Russia developed by Yandex researchers &amp; engineers. Used as part of key Yandex services including search, weather, and personal assistant</li><li>Model Quality: Better accuracy on several benchmarks vs. XGBoost, LightGBM, &amp; H2O</li><li>Speed: Faster training &amp; serving times on CPUs &amp; GPUs</li><li>Out-of-the-box Usability: Better model quality with default parameters, sophisticated categorical/text support and visual model analysis tools</li></ul><p>As with all data science &amp; machine learning problems you should set a baseline and try out a different techniques to see which one might be the best fit for your dataset but CatBoost paints an intriguing picture.</p><p>I had 2 quick objectives after reading about CatBoost:</p><ol><li>Run through an end-to-end example on AI Platform Notebooks using the dataset used in the <a href="https://cloud.google.com/ai-platform/training/docs/training-xgboost">XGBoost tutorial </a>(This can also be run for free on Google Colab with a few changes)</li><li>Put together a simple example of running CatBoost on AI Platform Training using both CPUs &amp; GPUs</li></ol><h3>AI Platform Notebook</h3><p>Notebook on <a href="https://github.com/mchrestkha/machine_learning_examples/tree/master/census/catboost/gcp_ai_platform/notebooks">GitHub HERE</a></p><p>If you love Jupyter Notebooks, <a href="https://cloud.google.com/ai-platform/notebooks/docs/create-new">AI Platform Notebooks</a> is Google Cloud’s managed JupyterLab Service. It takes minutes to get started and provides you with pre-built environments for your favorite frameworks, libraries and GPU drivers.</p><h4>Steps:</h4><ol><li>Create an AI Platform Notebook Instance with the <em>‘CUDA 10.1’</em> image with a Nvidia T4 GPU</li><li>Open up the JupyterLab UI</li><li>Open the Terminal and install CatBoost using <em>‘pip install catboost’</em>. All other libraries (pandas, numpy, sklearn) and drivers (cuda, nccl) are already installed as part fo the base image</li><li>Run through this <a href="https://github.com/mchrestkha/machine_learning_examples/blob/master/census/catboost/gcp_ai_platform/notebooks/catboost_census_notebook.ipynb">notebook</a></li></ol><h4>Observations:</h4><ul><li>CatBoost works with all your existing python &amp; cuda libraries so setup was a breeze. It took me &lt;5 min to get up and running</li><li>The CatBoost syntax and APIs are well designed and follow familiar patterns used by other libraries</li><li>GPUs were simple to use. I first used a prebuilt utility to check if the T4 GPU was being recognized.</li></ul><pre><strong>from</strong> <strong>catboost.utils</strong> <strong>import</strong> get_gpu_device_count<br>print(&#39;I see <strong>%i</strong> GPU devices&#39; % get_gpu_device_count())</pre><p>I then simply added <em>task_type=“GPU” </em>in the model definition and the model.fit automatically trained on the GPU.</p><p>**NOTE** In this example GPUs did not provide speedup over CPUs since the data was small (in both rows and features). I was however able to run through their <a href="https://github.com/catboost/tutorials/blob/master/tools/google_colaboratory_cpu_vs_gpu_tutorial.ipynb">GPU example with the epsilon</a> dataset (400K examples, 2K features) and saw a 5–6x speedup with the T4 GPU.</p><h3>AI Platform Training</h3><p>Training Package on <a href="https://github.com/mchrestkha/machine_learning_examples/tree/master/census/catboost/gcp_ai_platform">GitHub HERE</a></p><p>One of the main advantages of using a cloud provider is being able to leverage on-demand infrastructure only when you need it. AI Platform Training allows you submit code (or a docker container) to provision resources only for the time needed to process and train your data. Below are the steps I took to package the training code &amp; dependencies, submit the code to AI Platform, and save the model to Google Cloud Storage.</p><h4>Steps:</h4><ol><li><a href="https://cloud.google.com/sdk/docs/quickstarts">Install the Google Cloud SDK</a> on your development machine. If you are using Google Cloud Shell, Google Cloud Compute Engine, or the Terminal on the AI Platform Notebook instance in the previous section the SDK should already be pre-installed.</li><li><a href="https://cloud.google.com/storage/docs/creating-buckets">Create a Google Cloud Storage bucket</a> to save your model after the training is completed.</li><li>Package the training code in the following directory and file structure:</li></ol><pre><strong>root directory<br>-trainer directory<br></strong>--train.py<br>--__init__.py<br><strong>-scripts directory<br></strong>--train-cloud.sh<br>setup.py</pre><ul><li><em>train.py</em> file includes the same code from the Jupyter notebook example in the previous section</li><li><em>train-cloud.sh</em> includes the AI Platform SDK command to submit the training job</li><li><em>setup.py</em> file includes any python libraries to install on the on-demand server(s) that are not by default part of the hosted service. In this case you only need to specify ‘catboost’.</li></ul><p>4. Update the variables in train-cloud.sh and run the script.</p><pre>gcloud ai-platform jobs submit training $JOB_NAME \<br>--job-dir $JOB_DIR \<br>--package-path $TRAINING_PACKAGE_PATH \<br>--module-name $MAIN_TRAINER_MODULE \<br>--region $REGION \<br>--runtime-version=$RUNTIME_VERSION \<br>--python-version=$PYTHON_VERSION \<br>--scale-tier $SCALE_TIER</pre><p>5. The training job progress and status will be viewable at <a href="http://console.cloud.google.com/ai-platform/jobs">http://console.cloud.google.com/ai-platform/jobs</a> and the saved model (in my case ONNX format) will be saved in the appropriate Google Cloud Storage bucket.</p><h4>Observations:</h4><ul><li>While AI Platform Training only lists TensorFlow, XGBoost, &amp; SKLearn as officially supported hosted frameworks, CatBoost works seamlessly by including it in the setup.py file. You can also use <a href="https://cloud.google.com/ai-platform/training/docs/custom-containers-training">custom containers</a> to submit a job that has more complex runtime requirements.</li><li><strong>TODO</strong>: Need to add in a few custom arguments to refactor the code to support both CPUs &amp; GPUs automatically. Currently I have to manually comment in/out the GPU task_type in the train.py code.</li></ul><h3>Future follow-ups</h3><ul><li>Serve the model for both batch and online needs</li><li>Play with tuning options to maximize model quality</li><li>Play with the built-in visualization and explainability tools</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=130e4407e132" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>