<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by HK Lee on Medium]]></title>
        <description><![CDATA[Stories by HK Lee on Medium]]></description>
        <link>https://medium.com/@overpresentme?source=rss-d540409463e9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OQ-mD5DLbG2vrns0gAl1Ww.jpeg</url>
            <title>Stories by HK Lee on Medium</title>
            <link>https://medium.com/@overpresentme?source=rss-d540409463e9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 07:59:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@overpresentme/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[[ML] K8S + A100 MIG + wandb Sweep = Super-fast parameter tuning]]></title>
            <link>https://medium.com/@overpresentme/ml-k8s-a100-mig-wandb-sweep-super-fast-parameter-tuning-1410d35adb16?source=rss-d540409463e9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1410d35adb16</guid>
            <category><![CDATA[nvidia]]></category>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[wandb]]></category>
            <category><![CDATA[k8s]]></category>
            <dc:creator><![CDATA[HK Lee]]></dc:creator>
            <pubDate>Wed, 21 Sep 2022 15:40:43 GMT</pubDate>
            <atom:updated>2022-09-21T15:40:43.809Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*wmozgCuwxkiWu4xTBYOETw.png" /><figcaption>MIG of A100</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8UtHeF9I2kIDKKk7" /><figcaption>sweep of weight &amp; bias</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WnezNP5WMz52QefD.png" /><figcaption>k8s: container orchestration</figcaption></figure><h3>Introduction</h3><h4>k8s (Kubernetes)</h4><p>Container Orchestration System. With k8s, ML engineer can <strong>easily make a lot of training containers</strong>.</p><h4>A100 MIG (Multi-Instance GPU)</h4><p>If you have an A100 GPU, you can utilize one A100 GPU(80GB) as up to seven independent GPU Instance (10GB). It’s useful for the inference service or <strong>multiple training jobs</strong>.</p><h4>Sweep of wandb (weight &amp; bias)</h4><p>Sweep makes hyperparameter optimization easy. If search options (ex: lr=0.001~0.01) are decided once, you can <strong>do experiments under various conditions without specifying them in the code</strong>.</p><h3>Practice</h3><p>First, enable MIG in A100 GPU. If you don’t have A100 GPU, you can skip this stage (MIG &amp; additional k8s plugins).</p><p><a href="https://docs.nvidia.com/datacenter/tesla/mig-user-guide/">NVIDIA Multi-Instance GPU User Guide</a></p><p>If you have successfully followed the above user guide, you can find “MIG Enabled” when entering nvidia-smi.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*wYekELjFqitplXHKeOUG_w.png" /></figure><p>Next, suppose k8s cluster is builded up, several additional plugins should be installed for k8s to identify MIG resource.</p><p><a href="https://docs.nvidia.com/datacenter/cloud-native/kubernetes/mig-k8s.html">MIG Support in Kubernetes - NVIDIA Cloud Native Technologies documentation</a></p><p>According to the above documentation, <strong>k8s-device-plugin</strong> and <strong>gpu-feature-discovery</strong> should be installed.</p><p>In my case, eight A100 gpus become 56 gpu instances. And k8s have successfully identified MIG resource.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*6d0QGvLYGK8iHg4lHQdELg.png" /><figcaption>k8s identify MIG and 8 gpus become 56 mig-10GB</figcaption></figure><p>It’s time to train a lot.</p><p>You may be possible to do training under various conditions with programming (ex: python train.py --batch 64 --lr 0.001).</p><p>But with the sweep of wandb, this operation becomes super-easy.</p><p>First, make sweep.yaml like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/257/1*v20UFy_aUWhhtdXlqiJktg.png" /></figure><p>Then, create sweep-id with this yaml file. Now you can get various configuration from wandb server using this sweep-id (random / grid / bayes).</p><pre>wandb sweep sweep.yaml</pre><pre>python train.py &lt;sweep-id&gt;</pre><p>I would not explain the detail of this process. Please refer to the official documentation.</p><p>After all, I could easily find the best hyperparmeter options with little effort.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HlHvkFJjHwuEhRKUk_hYfw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/203/1*8kVsBK7B0o8RVixrXMJmJw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*KAt_AGYTEQOUg59r1GrwGg.png" /></figure><h3>Conclusion</h3><p>In this article, I have explained only hyperparameter tuning case. But in many cases, I believe this technique would ease bothersome problems in that utilizing many resources become super-easy.</p><p>I hope this article was helpful for you to happy ML.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1410d35adb16" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[Python] functools are slower than naive for-loop]]></title>
            <link>https://medium.com/@overpresentme/python-functools-are-slower-than-naive-for-loop-1b2b35df8d44?source=rss-d540409463e9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b2b35df8d44</guid>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[HK Lee]]></dc:creator>
            <pubDate>Tue, 20 Sep 2022 00:31:03 GMT</pubDate>
            <atom:updated>2022-09-20T00:33:57.297Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wxA-ub7F8eQATalt.jpeg" /></figure><p><a href="https://medium.com/codex/3-most-effective-yet-underutilized-functions-in-python-d865ffaca0bb">Don’t Run Loops in Python, Instead, Use These!</a></p><p>After reading the above article, I have tested the speed between reduce and for-loop.</p><pre>def sum_for_loop():</pre><pre>  arr = [1] * 101</pre><pre>  out = 0</pre><pre>  for x in arr:</pre><pre>    out += x</pre><pre>  return out</pre><pre>def sum_reduce():</pre><pre>  arr = [1] * 101</pre><pre>  out = reduce(lambda x,y: x+y, arr)</pre><pre>  return out</pre><pre>if __name__ == &#39;__main__&#39;:</pre><pre>  print(f&#39;for loop: {timeit.timeit(sum_for_loop)}&#39;)</pre><pre>  print(f&#39;reduce: {timeit.timeit(sum_reduce)}&#39;)</pre><p>The result was as follows.</p><pre>$ python test.py <br>for loop: 2.5328142<br>reduce: 4.7320765</pre><p><strong>reduce was twice slower than naive for-loop.</strong></p><p>I think (lambda) function call is unneccesarily repeated in reduce unlike naive for-loop.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b2b35df8d44" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>