<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nelson Punch on Medium]]></title>
        <description><![CDATA[Stories by Nelson Punch on Medium]]></description>
        <link>https://medium.com/@tomneo2004?source=rss-83260f508b12------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*hsWEvsWUPOxujWNT.jpg</url>
            <title>Stories by Nelson Punch on Medium</title>
            <link>https://medium.com/@tomneo2004?source=rss-83260f508b12------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:27:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tomneo2004/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[[UE5] Blending animation montage]]></title>
            <link>https://medium.com/software-dev-explore/ue5-blending-animation-montage-933f00adaa1e?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/933f00adaa1e</guid>
            <category><![CDATA[unreal-engine-5]]></category>
            <category><![CDATA[animation]]></category>
            <category><![CDATA[montage]]></category>
            <category><![CDATA[c-plus-plus-language]]></category>
            <category><![CDATA[blending]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Sun, 03 Aug 2025 03:59:51 GMT</pubDate>
            <atom:updated>2025-08-03T04:28:26.160Z</atom:updated>
            <content:encoded><![CDATA[<p>If you follow along <em>Stephen Ulibarri’s</em> course <a href="https://www.udemy.com/course/unreal-engine-5-the-ultimate-game-developer-course"><strong><em>Unreal Engine 5 C++ The Ultimate Game Developer</em></strong></a>, after you completed section 12 in the course, the character Echo will standing still while equipping/unequipping weapon. This is the feature course taught you. However I prefer to equip/unequip weapon while the character Echo is running.</p><p>To do achieve that, you need to blend equip/unequip animation montage in the character’s animation blueprint. Moreover it can only effect the character’s upper body only. Doing so we are able to play equip/unequip animation montage while maintaining the character’s lower body animation.</p><p>In my case, I want my Echo character to equip/unequip weapon while running but stand still while attacking.</p><p>This is the result.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FRwsZv87vQQ8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DRwsZv87vQQ8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FRwsZv87vQQ8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/98009da4676a3b49829817f01e1091c2/href">https://medium.com/media/98009da4676a3b49829817f01e1091c2/href</a></iframe><h3>1. Animation slot in montage</h3><p>Open Echo’s equip and unequip animation montage. Then open <strong>Anim Slot Manager</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*suWGh5QnabUX21l30keCWw.png" /></figure><p>Manager will show up at lower right corner. In the manager, create a new slot by clicking <strong>Add Slot</strong> button then enter slot name. For me I use name “UpperBody”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*mM6QD3RMcs7SlTRuFA3dBg.png" /></figure><p>In side animation montage track, there is a default slot name <strong>DefaultGroup.DefaultSlot</strong> under <strong>Montag (DefaultGroup)</strong>. That default one is we are using, now click the little triangle say slot when you hover it and change it to <strong>DefaultGroup.UpperBody </strong>in my case<strong> </strong>or your preferred slot name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0MeGAKv2zlJsIkkFofHiw.png" /></figure><p>Later this slot will be used in the character’s animation blueprint for blending.</p><h3>2. UpperBodyOnly variable</h3><p>Back to project’s code in visual studio. Open Slash character’s <strong>AnimInstance</strong> class’s header file. If you follow the course, the header file name would be <strong>SlashAnimInstance.h</strong>.</p><p>Create a boolean variable name <strong>UpperBodyOnly</strong> under protected and a setter for the variable as inline function under public. In addition expose <strong>UpperBodyOnly </strong>to blueprint<strong>.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0c6e7036db17921d415e9d087301b42/href">https://medium.com/media/f0c6e7036db17921d415e9d087301b42/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*14fG97ir1Ag_dwOJClh_wA.png" /></figure><p>The variable is used to determine whether an animation montage must be played on the character’s upper body only or not. If it is false then animation montage will be played on full body.</p><blockquote>The reason to do this is my preference as I want the character to play equip/unequip animation montage only for upper body but full body for any attacking animation montage.</blockquote><h3>3. Setup animation montage blending</h3><p>Open Echo the character’s animation blueprint. If you follow the course then it would be name <strong>ABP_Echo</strong>. In there open <strong>AnimGraph</strong>.</p><p>Modify the animation graph as follow. I will explain below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAjWXtEgd9QJrdz3rTjEbA.png" /></figure><p>Three graph node we are going to use.</p><ul><li><strong>Slot</strong>: The slot we setup in animation montage.</li><li><strong>Layered blend per bone</strong>: Blend animation base on bone, in this case we want upper body bones.</li><li><strong>Blend Poses by bool</strong>: Use this to determine whether use full body pose or blended pose.</li></ul><h4>Add UpperBody slot node</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*bUMoc_I0tTQI3-JacW6KcA.png" /></figure><p>Once the node is added it is set to <strong>DefaultSlot</strong> by default. Click on the slot node and in <strong>Detail panel </strong>on the right change it to <strong>UpperBody</strong> for <strong>Slot Name</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*qAmMuI9f_8xiATkwu1Rb7Q.png" /></figure><h4>Add <strong>Layered blend per bone node</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*pBW7ExTK1xw70OhEfVJIBg.png" /></figure><p>Click on the node and focus on detail panel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*V9kIqiwu_7bqWauMD3_edQ.png" /></figure><p>Under <strong>Config</strong> -&gt; <strong>Layer Setup</strong> click + icon button next to the <strong>Branch Filters</strong>. This will create a new filter in array. Expend the filter and configure the settings.</p><ul><li><strong>Bone Name</strong>: Which bone we want to use for blending, usually it would be the root bone of a chain. For instance, I want my character’s upper body to be used for blending then it would be particular spine bone which is root bone for entire upper body. Entire upper body children bones of the root bone would be used for blending. <strong>The root bone here is not the root of character’s skeleton</strong>.</li><li><strong>Blend Depth</strong>: Usually set to 0 or 1. If set to 3 then 3rd bone from the root bone and children bones of 3rd bone will be used for blending.</li></ul><p>Finding the root bone for blending is easy. If you use Echo’s skeleton then the name of first spine bone is <strong>spine_01</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dBjuVf4gsc-a0yyjRKrkcw.png" /></figure><h4>Add Blend Poses by bool node</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*wjuiQHPp_lYSPkOT-JlmqQ.png" /></figure><p>Once the node is added then hook <strong>Active Value</strong> to <strong>Upper Body Only</strong> variable, which we exposed it to blueprint from <strong>SlashAnimInstance.h</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/443/1*OPKkDJgvbO8NLWeejLrcBA.png" /></figure><p>To get the variable search for the variable name under graph node searching prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*rBxsToTsus6X2Mo2d_lt4g.png" /></figure><h4>Blending animation</h4><p>First work on upper body blending.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*nE3kb6PYZuwKlstNOT7pqQ.png" /></figure><p><strong>Layered blend per bone</strong> node take <strong>Base Pose</strong> as source and then blend with <strong>Blend Pose 0</strong>. For example, the base pose is playing running animation, then blend with equip animation sequence in equip and unequip animation montage. Since we set <strong>Layered blend per bone</strong> node’s filter bone name to spine_01 then spine_01 bone and its children bone will be effect by equip animation. Running + Equipping = Running while Equipping.</p><p>Secondly work on full body animation. Because we want attacking to be full body animation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*u-wBkQFx0CadTK2B4kyEig.png" /></figure><p>If you look into attacking animation montage then it is under <strong>DefaultSlot</strong>.</p><p>Finally determine upper body or full body animation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*utPsO_DqkYEskfdARVSQGg.png" /></figure><p>When Upper Body Only variable is true then play upper body blended pose otherwise play full body pose.</p><h4>4. <strong>Manipulate UpperBodyOnly variable</strong></h4><p>Open project’s code in visual studio. We need to change UpperBodyOnly variable’s value so Echo the character can use correct pose.</p><p>We will change the variable’s value through its setter. Open Echo the character’s class cpp file. If you follow along the course it is named<strong> SlashCharacter.cpp</strong>. Two places we need to change the variable’s value.</p><ul><li><strong>PlayAttackMontage</strong></li><li><strong>PlayEquipMontage</strong></li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e6f30d2bb49c379d9866fcf9602f3381/href">https://medium.com/media/e6f30d2bb49c379d9866fcf9602f3381/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QN1l2l_kHfMtgWhhQDbLrw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8I_ptN4pek0Bx6XprkSvQ.png" /></figure><p>Set the variable’s value to false then playing attacking montage but set to true when playing equip or unequip montage.</p><p>There is a place need to be changed, which is character’s movement callback function. If we don’t change it then Echo the character still not able to move while equip/unequip but the character’s final blending pose is correct.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1f4fdb7222d373bb7ac914788b2be259/href">https://medium.com/media/1f4fdb7222d373bb7ac914788b2be259/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*upkG2BTmUzwOB8iTcKG7Gw.png" /></figure><p>Remember EActionState enum?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/95dadef6ad206d8df6261f10d965d7f0/href">https://medium.com/media/95dadef6ad206d8df6261f10d965d7f0/href</a></iframe><p>At beginning of function, the one comment out is the old one; don’t move the character only if the character’s action state is not equal to unoccupied, that mean any action will prevent the character moving including equip and unequip state.</p><p>We need to change to check if the character’s action state is equal to attacking only, this mean don’t move the character when action state is only in attacking state, it exclude equip, unequip, unoccupied state.</p><h3>Conclusion</h3><p>This is the result of my goal which allow the character to equip/unequip when running.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FRwsZv87vQQ8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DRwsZv87vQQ8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FRwsZv87vQQ8%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/98009da4676a3b49829817f01e1091c2/href">https://medium.com/media/98009da4676a3b49829817f01e1091c2/href</a></iframe><h3>References:</h3><ul><li><a href="https://dev.epicgames.com/documentation/en-us/unreal-engine/using-layered-animations-in-unreal-engine">https://dev.epicgames.com/documentation/en-us/unreal-engine/using-layered-animations-in-unreal-engine</a></li><li><a href="https://www.youtube.com/watch?v=bU95pf5hTkI">https://www.youtube.com/watch?v=bU95pf5hTkI</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=933f00adaa1e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/ue5-blending-animation-montage-933f00adaa1e">[UE5] Blending animation montage</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Counting exercise repetition with deep learning]]></title>
            <link>https://medium.com/software-dev-explore/counting-exercise-repetition-with-deep-learning-ee5a38df9297?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/ee5a38df9297</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[signal-processing]]></category>
            <category><![CDATA[exercise]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Sat, 22 Jun 2024 07:27:49 GMT</pubDate>
            <atom:updated>2024-07-12T05:20:26.195Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*LEHlXmhPtcfF81dn" /><figcaption>Photo by <a href="https://unsplash.com/@pedroaraujo74?utm_source=medium&amp;utm_medium=referral">Pedro Araújo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>I love exercise and improve my whole body strength. I do weightlifting and body weight exercise weekly.</p><p>Volume help your body to grow and increase your strength gradually. Volume = Weight + Repetition.</p><p>From time to time you might lost how many repetition you have done while focusing on performing a challenging exercise. At least it happen to me sometime. Therefore I start to have an idea long ago on how to use artificial intelligent to assist physical exercise? Using deep learning to count how many repetition you just perform for an exercise.</p><p>I was struggle in the beginning on how to solve this problem. After a long research and reading articles, I found a viable but not perfect solution. And here are some of my progress.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*frmIr0hehg-ScZa0s4AtqQ.gif" /><figcaption>jumping jacks</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A7qby0I0OTENxUimatv3iQ.gif" /><figcaption>push ups</figcaption></figure><h3>Problems &amp; Details</h3><p>There are two main problems I need to solved.</p><p><strong>The first immediate problem is how to track human joints from video?</strong></p><p>Most exercises can be recognized by two factors, one is joint to joint distance and another is joint rotation. For example exercise push ups can be recognized by rotation of elbow joint and shoulder to wrist distance. This is why tracking human joints in real-time is important.</p><p><strong>Second problem is how do I translate those tracked joints’ movement into a recognizable pattern?</strong></p><p>The answer is distance and rotation of joints during <strong>eccentric </strong>and <strong>concentric </strong>exercise movement. Take a look at image below about eccentric and concentric movement for squat exercise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*VR1xSDluxS5IC6gQj0e_2Q.gif" /><figcaption>Squat</figcaption></figure><p>From the image I can see some information below about squat exercise.</p><ol><li>Eccentric: From standing position to position where you have knee bending 90 degree and hip almost touch your ankle. I also call this bottom position.</li><li>Concentric: From bottom position back to standing position.</li></ol><p>Performing both eccentric and concentric movement number of time to achieve your desired repetition.</p><p>What I need to pay attention here are distance between hip to ankle and rotation of knee because this movement form a noticeable pattern in each movement cycle.</p><h3>Methods for counting repetition</h3><p>From this <a href="https://towardsdatascience.com/vision-based-rep-counting-in-the-wild-cb9a4d1bdb7e"><strong><em>article </em></strong></a>there are number of methods I can use to solve this problem. However each one of them comes with pro and con.</p><ul><li><a href="https://research.google/blog/repnet-counting-repetitions-in-videos/"><strong><em>RepNet</em></strong></a></li><li>Rule-based using <a href="https://blog.cult.fit/posts/winning-interactivity-using-computer-vision"><strong><em>Post Estimation</em></strong></a></li><li>Signal processing</li><li><a href="https://smashlab.io/publications/gymcam/"><strong><em>GymCam</em></strong></a></li><li><a href="https://towardsdatascience.com/how-i-created-the-workout-movement-counting-app-using-deep-learning-and-optical-flow-89f9d2e087ac"><strong><em>Optical Flow</em></strong></a></li></ul><p>Although each method seems interesting, I choose signal processing method over others due to its simplicity and efficiency. Beside it is accuated enough. As <a href="https://towardsdatascience.com/vision-based-rep-counting-in-the-wild-cb9a4d1bdb7e"><strong><em>article</em></strong></a><strong><em> </em></strong>had point out some cons, non-generic, sensitive to background noise, need the zero-crossing line from reference video.</p><h3>Generate exercise statistics</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A7qby0I0OTENxUimatv3iQ.gif" /></figure><p>See the red line on the graph at right window? That is the <strong><em>mean</em></strong> for this push ups exercise. I can use this <strong><em>mean </em></strong>for counting repetition. Following are steps to generate mean from an input video.</p><h4>Tracking human joints</h4><p><a href="https://www.kaggle.com/models?query=movenet&amp;tfhub-redirect=true"><strong><em>MoveNet </em></strong></a>is a deep learning model that can track 17 key points for human body. It is accuracy and fast, in addition it is small in model size. All I need to do is to capture each frames from video and feed them into <a href="https://www.kaggle.com/models?query=movenet&amp;tfhub-redirect=true"><strong><em>MoveNet</em></strong></a><strong><em> </em></strong>and then<strong><em> </em></strong>the model will produce 17 key points for each frames.</p><h4>Define metric for exercise</h4><p>Signal processing is specific to the exercise and non-generic therefore I need to define what metrics a particular exercise need. Metric here means distance between joints or rotation of a joint. Take squat for example I would define metrics below.</p><ul><li>lshl-lhip-dist: left shoulder to left hip distance</li><li>rshl-rhip-dist: right shoulder to right hip distance</li><li>lhip-lakl-dist: left hip to left ankle distance</li><li>rhip-rakl-dist: right hip to right ankle distance</li><li>lhip-angle: left hip angle</li><li>rhip-angle: right hip angle</li><li>lknee-angle: left knee angle</li><li>rknee-angle: right knee angle</li></ul><p>Key points I get from <a href="https://www.kaggle.com/models?query=movenet&amp;tfhub-redirect=true"><strong><em>MoveNet</em></strong></a><strong><em> </em></strong>I then calculate these metrics. <strong>Aware of camera angle before calculation</strong>. Since I have no idea what angle camera will facing, the camera could be face at front or side. Therefore I have to make assumption. For squat I would assume camera is on right/left side then I would make calculation on y axis. For instance knee rotation can only be measure from left/right side not at front.</p><h4>Lowpass filter</h4><p><a href="https://dobrian.github.io/cmp/topics/filters/lowpassfilter.html"><strong><em>Lowpass filter</em></strong></a> is a signal processing technique and is used to filter out high frequency signals.</p><p>I treat each metrics as individual signal and they are not always come out as smooth as I expected. A example below is an image of high frequency signals and signals after lowpass filter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*-xN5ZRF7AqDmde6elaOc0w.png" /><figcaption>source: <a href="https://x-engineer.org/wp-content/uploads/2021/07/Low-pass-filter-example-signal.png">https://x-engineer.org/wp-content/uploads/2021/07/Low-pass-filter-example-signal.png</a></figcaption></figure><p>Metrics are similar to the purple signal. I can see a lot of jittering. The signal after lowpass filter is the one in blue.</p><p>I need to do lowpass filter for each metrics to get rid of jittering.</p><p>The simplest way to do lowpass filter is using following formula.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/188/1*gvJnzt7YYiY7AjlAs1gCBg.jpeg" /></figure><ul><li>yn: Output signal</li><li>xn: Current input signal</li><li>xn-1: Previous input signal</li><li>a: Coefficient value (Alpha) where the range is between 0 ~ 1. If Alpha is 1 then output signal is equal to input signal,</li></ul><h4>Remove stationary metrics</h4><p>Some metrics are representative and some are not. By representative I mean frequency of signals in a metric are variant(non-stationary) whereas non-representative means frequency of signals in a metric are non-variant(stationary).</p><p>Variant signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*waJF9cvM4LVzDvuUoncrUg.png" /><figcaption>Frequency variant signals</figcaption></figure><p>None-variant signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*aaxW6BvuHujw2kBh3lDbGg.jpeg" /><figcaption>Almost flatten</figcaption></figure><p>The way to determine whether to keep a metric or not is to use its <strong><em>standard deviation</em></strong>. I can set a desired threshold for standard deviation and calculate standard deviation for a metric then I compare them together to decide I should keep this metric or not.</p><h4>Aggregate metrics &amp; save statistics</h4><p>After all non-stationary metrics are selected then I can add them together to become one sequential signal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nfqwKcJ_ZNYAUKP1KtYUnw.png" /><figcaption>Aggregation of all non-stationary signals for jumping jack</figcaption></figure><p>Now I can save non-stationary metrics’ name and value of mean in a json file. Of course other information can be saved for further use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*CPklHHSyMtB3PSgADd0xNw.png" /><figcaption>json file contain exercise reference</figcaption></figure><p>As above image you can see json contain 2 exercises and there is <strong><em>motion_names</em></strong> contain names of non-stationary metrics. And some statistics under <strong><em>reference</em></strong>.</p><h3>Real-Time repetition counting</h3><p>For real-time repetition counting, we are still doing the same steps that I mention above and repetition counting at final steps. The flow look like below.</p><p>Get 17 key points -&gt; Calculate metrics -&gt; Lowpass filter -&gt; Remove all stationary metrics -&gt; Aggregate non-stationary metrics -&gt; Repetition counting.</p><p>Only two steps I need to pay attention to Remove all stationary metrics and Repetition counting.</p><h4>Remove all stationary metrics</h4><p>I don’t need to calculate it again since metrics’ name are already saved in json file. I could simply get these metric names and remove all metrics that I don’t needed for a particular exercise.</p><h4>Repetition counting</h4><p>First we retrieve value mean from json file for a certain exercise.</p><p>Now how do I know a repetition had been performed? Take a look at image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HfVFCq_tbibxhGGSF903RA.png" /></figure><p>There are 2 colours green and red in the image and they represent down state(green) and up state(red). These 3 dots are data points in the entire signal as well.</p><p>To count repetition I can just observe the pattern. If last data point is above mean and current data point is below mean then we say there is 1 repetition completed. Some problem with this method.</p><ul><li>Need a proper sliding window size in order to detect repetition.</li><li>Unable to find total repetition from the entire signal.</li></ul><p>To solve these two problems, I had done some research about pattern recognition such as <a href="https://matrixprofile.org/"><strong><em>Matrix Profile</em></strong></a>. I end up a simple method by myself.</p><p>Why don’t I convert signal into 0 and 1. Make any data point that is below mean as 0 and above as 1. I will call this converted signal as binary signal. From above signal image we could end up with</p><p>[1100011100] to the first greed dot. Now make sequential 1 and 0 to become 1 digit only and we have</p><p>[1010]. Then I can define a pattern sliding window look like this [010]. Finally slide window through entire binary signal. For this example only 1 pattern have been found.</p><ul><li>Use combination of 0 and 1 to define sliding window size.</li><li>Able to find total repetition from signal.</li><li>Signal length had been reduced.</li></ul><h3>Conclusion</h3><p>Exercise repetition counting with signal processing can be helpful for people want to tracking their repetition during exercise. This method leverage deep learning and signal processing to achieve repetition counting and it is fairly accurate. Furthermore it can still perform counting while partial human body is occluded.</p><p>On production level it is fast and efficient enough. Of course it depend on the spec of machine it run on. Below is application integrated with exercise repetition counting.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FvTK90y-4Dac%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DvTK90y-4Dac&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FvTK90y-4Dac%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b4ef4beffb1baa1d0bc938f64309ffcc/href">https://medium.com/media/b4ef4beffb1baa1d0bc938f64309ffcc/href</a></iframe><h3>Application download</h3><p>I had made an application for this exercise repetition counting. The application is capable of repetition counting from video clip source and webcam.</p><p>If you would like to try it out here are links to download.</p><ul><li><a href="https://mega.nz/folder/WJhi0ZTK#eWCuymKMZLN0NJTfE995Bg"><strong><em>Link</em></strong></a></li></ul><h3>Source Code</h3><p>All source code is hosted on Github <a href="https://github.com/NPMachineLearning/SmartTrainer/tree/master"><strong><em>here</em></strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee5a38df9297" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/counting-exercise-repetition-with-deep-learning-ee5a38df9297">Counting exercise repetition with deep learning</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WSL2 — Webcam]]></title>
            <link>https://medium.com/software-dev-explore/wsl2-webcam-23c1dc9408ae?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/23c1dc9408ae</guid>
            <category><![CDATA[webcam]]></category>
            <category><![CDATA[wsl-2]]></category>
            <category><![CDATA[linux-kernel]]></category>
            <category><![CDATA[ubuntu]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Fri, 19 Apr 2024 09:11:50 GMT</pubDate>
            <atom:updated>2024-06-22T07:40:20.971Z</atom:updated>
            <content:encoded><![CDATA[<h3>Tensorflow WSL2 — Webcam</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f_tgR7HjOgIlN2YY" /><figcaption>Photo by <a href="https://unsplash.com/@ethanchoover?utm_source=medium&amp;utm_medium=referral">Ethan Hoover</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>I wrote an article about setting up Tensorflow with GPU on WSL2 <a href="https://medium.com/software-dev-explore/tensorflow-with-wsl-f06651df25b7"><strong><em>here</em></strong></a>. Now I am encounter a problem which I am not able to use my webcam. However after a days of search and experiments, I finally got it working.</p><p>Here I will write down how I do it step by step.</p><h3>Attach webcam to WSL2</h3><p>In order to expose our webcam to WSL2, we need a solution. According to documentation <a href="https://learn.microsoft.com/en-us/windows/wsl/connect-usb"><strong><em>here</em></strong></a> <a href="https://github.com/dorssel/usbipd-win"><strong><em>usbipd-win</em></strong></a> can help us to overcome this problem.</p><blockquote><a href="https://github.com/dorssel/usbipd-win"><strong><em>usbipd-win</em></strong></a><em> </em>is a windows software for sharing locally connected USB devices to other machines, including Hyper-V guests and WSL 2.</blockquote><blockquote>Following commands are on windows</blockquote><p>You can download latest version either from <a href="https://github.com/dorssel/usbipd-win/releases">here</a> or use winget on windows.</p><pre>winget install usbipd</pre><p>To list connected usb device.</p><pre>usbipd list</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*2m4C-LoKhcRIejgt6lAPQQ.jpeg" /></figure><p>First we need to bind device. Once a device is bind, you don’t need to bind it again. For example I would like to use <strong><em>Integrated Webcam</em></strong>.</p><blockquote>Make sure you run your terminal as administrator otherwise it will not allow you to do binding.</blockquote><pre>usbipd bind -b 1-5</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*3lujmcfGbk17pM8QlhKyrg.jpeg" /></figure><p>Then you can see its <strong><em>STATE</em></strong> is now <strong><em>Shared</em></strong>.</p><p>Next is to attach the device to WSL2.</p><pre>usbipd attach -w -b 1–5</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*iOzNk0H7zpXRNm5Tpoyg4w.jpeg" /></figure><blockquote>You need to make sure the distribution in WSL2 is running in order to attach the device to it. The way to do it is to start distribution in another terminal.</blockquote><blockquote>Following commands are on WSL2</blockquote><p>To check if webcam is attached.</p><pre>lsusb</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/1*6mdamUk3QKUfYPWmpTGW2w.jpeg" /></figure><p>Yes it is attached to WSL2.</p><h3>Build WSL2 Linux Kernel from scratch</h3><p>In order to grab video from the webcam by attaching a webcam to WSL2 is not enough. At this point of time I am writing the article, the WSL2 kernel version is 5.15.150.1 which doesn’t included proper driver for webcam.</p><p>The only way I found to solve this problem is to build a new kernel and include the proper drivers.</p><h4>Preparation</h4><blockquote>Following commands are on windows</blockquote><p>First update WSL2.</p><pre>wsl --update</pre><blockquote>Following commands are on WSL2</blockquote><p>Next update and upgrade apt then install necessary dependencies.</p><pre>sudo apt update &amp;&amp; sudo apt upgrade -y &amp;&amp; sudo apt install -y build-essential flex bison dwarves libssl-dev l<br>ibelf-dev libncurses-dev pkg-config </pre><p>Check your version of WSL2.</p><pre>uname -r</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*4KJ4lBHmC7w08BNmChxyrg.jpeg" /></figure><p>For me it is 5.15.150.1</p><p>Clone WSL2 Linux Kernel repo from <a href="https://github.com/microsoft/WSL2-Linux-Kernel"><strong><em>here</em></strong></a><strong><em> </em></strong>and select the branch that match your version.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*73bjekJfmGXAHT5FZ4FvAQ.jpeg" /></figure><pre># install git if you haven&#39;t<br>sudo apt install -y git<br><br># clone the repo<br>git clone https://github.com/microsoft/WSL2-Linux-Kernel.git</pre><h4>Configuration for building kernel</h4><blockquote>Following commands are on WSL2</blockquote><p>After it the repo finish downloaded, we need to prepare configuration file to build the kernel.</p><pre># go to the folder<br>cd WSL2-Linux-Kernel/<br><br># copy current system configuration zip file to the folder<br>sudo cp /proc/config.gz config.gz<br><br># unzip file<br>sudo gunzip config.gz<br><br># rename unzipped config file to .config<br># .config file will be used as default configuration <br># file for building kernel<br>sudo mv config .config</pre><p>Then we start to configure what drivers we need to be included.</p><pre>sudo make menuconfig</pre><p>It will prompt a configuration window.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Lk-jBI8XnNadnntx33tlg.jpeg" /></figure><p>Make sure you select following drivers.</p><blockquote>When you are doing selecting, make sure you select it with <strong>*</strong> not <strong>M</strong>.</blockquote><pre><br># Build WSL2 kernel with usb camera support<br># menuconfig -&gt; Device Drivers -&gt; Multimedia support -&gt; Filter media drivers<br>#            -&gt; Device Drivers -&gt; Multimedia support -&gt; Media device types -&gt; Cameras and video grabbers<br>#            -&gt; Device Drivers -&gt; Multimedia support -&gt; Video4Linux options -&gt; V4L2 sub-device userspace API<br>#            -&gt; Device Drivers -&gt; Multimedia support -&gt; Media drivers -&gt; Media USB Adapters -&gt; USB Video Class (UVC)<br>#            -&gt; Device Drivers -&gt; Multimedia support -&gt; Media drivers -&gt; Media USB Adapters -&gt; UVC input events device support<br>#            -&gt; Device Drivers -&gt; Multimedia support -&gt; Media drivers -&gt; Media USB Adapters -&gt; GSPCA based webcams<br></pre><h4>Build kernel</h4><blockquote>Following commands are on WSL2</blockquote><p>After you finish configuration then we can start to build kernel.</p><pre># build kernel <br># $(nproc) return number of processors<br>sudo make -j $(nproc)<br>sudo make modules_install -j $(nproc)<br>sudo make install -j$(nproc)</pre><p>If you encounter bc: not found error like image below, you need to install <strong><em>bc</em></strong> by sudo apt install -y bc then rebuild kernel again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*7-bwWq8wXKvZRahv94jsLw.jpeg" /></figure><blockquote>A little tip here. After build process either completed or fail, there are some files will be generated. In order to make a clean rebuild we can run these commands to clean it up. Then you start from where you copy configuration file step.</blockquote><pre>sudo make clean<br>sudo make mrproper<br>sudo make distclean</pre><h4>Install kernel</h4><p>Once it is finished, you can see a file name <strong><em>vmlinux </em></strong>in the directory. That is your kernel. We need to copy kernel to windows system then tell WSL2 to find the kernel.</p><p>Copy kernel to windows system.</p><pre># replace /d/WSLDistri/Kernel/dist/kernel-5.15.150.1<br># with your desired directory path on windows<br># kernel-5.15.150.1 is file name<br>sudo cp vmlinux /mnt/d/WSLDistri/Kernel/dist/kernel-5.15.150.1</pre><p>Tell WSL2 where to find the kernel</p><p>first we need to create a file <strong><em>.wslconfig</em></strong> under your username’s folder on windows so that WSL2 will use it for configuration.</p><pre># replace [username] with your username<br># on window<br>vim /mnt/c/Users/[username]/.wslconfig</pre><p>This will enter text editor(vim). Next press i on keyboard to start editing and add following.</p><pre>[wsl2]<br># replace D:\\WSLDistri\\Kernel\\dist\\kernel-5.15.150.1<br># with your kernel location on windows<br># double backward slash for escaping is must<br>kernel=D:\\WSLDistri\\Kernel\\dist\\kernel-5.15.150.1</pre><p>When finish press ESC on keyboard to exit editing mode then shift + : for enter command. Enter wq to save and quit and press Enter.</p><p>Finally exit WSL2 and shut it down. You can exit WSL2 by typing exit or Ctrl + D on keyboard.</p><blockquote>Following commands are on windows</blockquote><pre>wsl --shutdown</pre><p>And then we start WSL2 with webcam attached to it.</p><pre>usbipd attach -w -b 1-5</pre><h3>Test</h3><p>Let’s see if webcam is working.</p><p>Check if webcam is attached to WSL2</p><pre>lsusb</pre><p>This will install and start webcam application.</p><pre>sudo apt install v4l-utils guvcview<br>sudo guvcview</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cMPp5e2Ke6CBULdUXWfN-g.jpeg" /></figure><p>Webcam is working. However the quality is low. For me low quality issue can be solved by switching to different webcam or lower resolution.</p><h3>Webcam doesn’t work</h3><p>Not all resolution, FPS and output type can works without problem. You might have this problem. And frame just freeze there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*VKe6Slj0p2oSAXX3Uu6p7A.jpeg" /></figure><p>This could be the FPS and resolution or output type doesn’t support. You can see the format by.</p><pre>v4l2-ctl --list-formats-ext</pre><p>and use.</p><pre>sudo guvcview</pre><p>To find out which FPS, resolution and output type works for you.</p><h3>Conclusion</h3><p>I went through using <a href="https://github.com/dorssel/usbipd-win"><strong><em>usbipd-win</em></strong></a><strong><em> </em></strong>to attach webcam to WSL2 and building kernel, which included drivers we want from scratch. And finally I test it.</p><p>This is a long process but it is not as complicated as I thought. In general, the webcam works without issue.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23c1dc9408ae" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/wsl2-webcam-23c1dc9408ae">WSL2 — Webcam</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tensorflow with WSL2]]></title>
            <link>https://medium.com/software-dev-explore/tensorflow-with-wsl-f06651df25b7?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/f06651df25b7</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[gpu]]></category>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[wsl-2]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Tue, 16 Apr 2024 09:24:49 GMT</pubDate>
            <atom:updated>2024-04-19T01:24:43.002Z</atom:updated>
            <content:encoded><![CDATA[<h3>Tensorflow GPU with WSL2</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G7NPsGZGBk-kyUr9" /><figcaption>Photo by <a href="https://unsplash.com/@martino_pietropoli?utm_source=medium&amp;utm_medium=referral">Martino Pietropoli</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>This is a walkthrough on how to make Tensorflow work in WSL2 and detect my graphics card(GPU).</p><p>I had tried official Tensorflow in WSL2 installation <a href="https://www.tensorflow.org/install/pip#windows-wsl2"><strong><em>here</em></strong></a>. And it didn’t work for me. In addition I have seen many people had same problem and ask in internet.</p><p>After pulling my hair and research for few days, I have made it works. At least in my case. <strong><em>The whole point in this walkthrough is to enable Tensorflow to detect my graphics card</em></strong>.</p><p>The reason to use WSL2 instead of Windows is because Tensorflow support GPU for Windows stopped at version 2.10 and any version after that will require you to use WSL2. This is what they said officially.</p><blockquote><strong>Caution:</strong> TensorFlow <strong>2.10</strong> was the <strong>last</strong> TensorFlow release that supported GPU on native-Windows. Starting with TensorFlow <strong>2.11</strong>, you will need to install <a href="https://tensorflow.org/install/pip#windows-wsl2">TensorFlow in WSL2</a>, or install <strong>tensorflow</strong> or <strong>tensorflow-cpu</strong> and, optionally, try the <a href="https://github.com/microsoft/tensorflow-directml-plugin#tensorflow-directml-plugin-">TensorFlow-DirectML-Plugin</a></blockquote><h3>My PC specs</h3><h4>Hardware</h4><ul><li>CPU: Intel(R) Core(TM) i7–8750H CPU @ 2.20GHz 2.21 GHz</li><li>RAM: 16.0 GB</li><li>Graphic Card: NVIDIA GeForce GTX 1060 with Max-Q Design(6GB)</li></ul><h4>OS</h4><ul><li>Windows 10 pro — 22H2–19045.4291</li></ul><h3>Before it start</h3><p>Make sure you update your Windows. <strong><em>Windows 10 is required and at least OS build 19044 or higher</em></strong>. It was written on official <a href="https://www.tensorflow.org/install/pip#windows-wsl2"><strong><em>website</em></strong></a>.</p><blockquote><strong>Note:</strong> TensorFlow with GPU access is supported for WSL2 on Windows 10 19044 or higher. This corresponds to Windows 10 version 21H2, the November 2021 update. You can get the latest update from here: <a href="https://www.microsoft.com/software-download/windows10">Download Windows 10</a>. For instructions, see <a href="https://docs.microsoft.com/windows/wsl/install">Install WSL2</a> and <a href="https://docs.nvidia.com/cuda/wsl-user-guide/index.html">NVIDIA’s setup docs</a> for CUDA in WSL.</blockquote><p>Make sure you have NVIDIA dedicate graphics card. You can check if your graphics card is supported <a href="https://developer.nvidia.com/cuda-gpus"><strong><em>here</em></strong></a>.</p><p>Make sure your NVIDIA dedicate graphics card is up-to-date. You can check it by running nvidia-smi in command-line on Windows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*AC5DCZ7jMhY0wPhIYWix1Q.jpeg" /></figure><h3>WSL and Preparation</h3><p>WSL stand for <strong><em>Windows Subsystem for Linux</em></strong>. In simple explanation, it is a Linux operating system under Windows.</p><p>To enable WSL on Windows 10/11 see <a href="https://www.ridom.de/u/Windows_Subsystem_For_Linux.html#Enabling_WSL_1_.28recommended_for_Windows_10.29"><strong><em>here</em></strong></a>.</p><p>Here I am going to install Ubuntu distribution. According to this document <a href="https://learn.microsoft.com/en-us/windows/wsl/install#prerequisites"><strong><em>here</em></strong></a>. <strong><em>You must be running Windows 10 version 2004 and higher (Build 19041 and higher)</em></strong>.</p><p>First launch either <strong>PowerShell </strong>or <strong>Command-line</strong> in Windows with <strong>Administrator</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*wtH0s6s9vsTBqsVoOCOUYw.jpeg" /></figure><p>Update WSL wsl --update then wsl --install this will install latest Ubuntu distribution. The reset is following setup instruction.</p><p>To start Ubuntu run wsl then it will start Ubuntu and you can start operate system with Linux commands. To exit run exit then you will back to Windows command-line. To shutdown wsl --shutdown .</p><p>The default working directory every time you start Ubuntu is /mnt/c/Users/[your Ubuntu username] where mnt is mounting point, c is <strong><em>C</em></strong> partition on disk, Users is <strong><em>Users</em></strong> folder on Windows.</p><p>In fact, this is the same directory of your user working directory on Windows. If you run ls On Ubuntu to display a list of files and folders, they are as same as the one on Windows C:\Users\[username]\. It also means that creating or deleting file either in Ubuntu or Windows will effect another system.</p><p>To test it run mkdir test which create an empty folder. Now you can find it under C:\Users\[username]\ on Windows. Delete it on Windows then run ls on Ubuntu. The folder is gone.</p><p>Personally I would like to work in home directory on Ubuntu. Every time I started Ubuntu I will go to home directory by running cd ~/ this will put you under /home/[username]/ and with it I can sure that anything I do is only on Ubuntu not on Windows.</p><h3>Tensorflow GPU on WSL2-Ubuntu</h3><p>From now on we will work on Ubuntu with Linux commands. Therefore make sure you run wsl start Ubuntu on Windows command-line. <strong><em>Also make sure you are at home directory</em></strong> by running cd ~/ on Ubuntu.</p><p>To see which directory are you in, you run pwd.</p><p>We will go through.</p><ul><li>Python3 and pip installation.</li><li>Create working directory and environment.</li><li>Install Tensorflow.</li><li>Install CUDA Toolkit.</li><li>Verify CUDA installation.</li><li>Install cuDNN.</li><li>Solve last puzzle and enable Tensorflow to detect GPU.</li></ul><h4>Install Python3 and pip</h4><p>Run</p><pre>sudo apt-get update &amp;&amp; sudo apt-get install python3 python3-pip</pre><p>. This will install Python3 programming language and pip a Python package manager.</p><h4>Create working directory &amp; environment</h4><p>Run</p><pre>mkdir tftest &amp;&amp; cd tftest</pre><p>Now we will work in this directory.</p><p>For creating environment I use <a href="https://virtualenv.pypa.io/en/latest/installation.html"><strong><em>virtualenv</em></strong></a>. Run</p><pre>python3 -m pip install virtualenv</pre><p>To create an environment in current directory</p><pre>python3 -m virtualenv .venv</pre><p>which create a folder name <strong><em>.venv</em></strong><em> </em>and all future <em>pip</em> installed libraries will be in that folder.</p><p>In order to work in the environment we need to activate it. To activate environment source .venv/bin/activate . To deactivate environment deactivate.</p><blockquote>Why environment? Instead of installing libraries in operating system, we install libraries in an isolated environment and the benefit of it is to not effect operating system. If something went wrong in the environment, operating system will not be damaged and all we need to do is to delete environment and recreate it again.</blockquote><h4>Install Tensorflow</h4><p>Base on document, to install latest Tensorflow that support GPU for WSL2 pip install tensorflow[and-cuda] while you are in the environment.</p><p><strong>Now it is important</strong>. Let’s see this <a href="https://www.tensorflow.org/install/source#gpu"><strong><em>table</em></strong></a>. You see the latest version is <strong><em>tensorflow-2.16.1</em></strong> while I am writing this walkthrough. Next make sure our python3 version is in <strong><em>3.9–3.12 </em></strong>by running python3 --version . Finally remember <strong>cuDNN </strong>and <strong>CUDA </strong>version. For me it requires both 8.9 and 12.3.</p><h4>Install CUDA Toolkit</h4><p>For CUDA. We will install version 12.3.</p><p>According to the setup <a href="https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2"><strong><em>document</em></strong></a><strong><em> </em></strong>for CUDA Toolkit at <strong><em>section 3 option 1</em></strong> we can download CUDA Toolkit from <a href="https://developer.nvidia.com/cuda-downloads?target_os=Linux&amp;target_arch=x86_64&amp;Distribution=WSL-Ubuntu&amp;target_version=2.0&amp;target_type=deb_local"><strong><em>here</em></strong></a>. <strong><em>However at this point I am writing this walkthrough it take me to version 12.4 which is not what I want</em></strong>.</p><p>To get the correct version, scroll down to <strong><em>Resource </em></strong>section at bottom of page and click <a href="https://developer.nvidia.com/cuda-toolkit-archive"><strong><em>Archive of Previous CUDA Releases</em></strong></a>. Find and click <a href="https://developer.nvidia.com/cuda-12-3-0-download-archive"><strong><em>CUDA Toolkit 12.3.0</em></strong></a><strong><em>. </em></strong>Select target platform.</p><ul><li>Operating System : Linux.</li><li>Architecture : x86_64.</li><li>Distribution : <strong>WSL-Ubuntu</strong> not Ubuntu. WSL-Ubuntun do not include NVIDIA driver whereas Ubuntu included NVIDIA drive. And in <a href="https://docs.nvidia.com/cuda/wsl-user-guide/index.html"><strong><em>this</em></strong></a> user guide at section 2.1 it says <strong><em>Do not install any Linux display driver in WSL</em>.</strong></li><li>Version : 2.0</li><li>Installer Type : Select any one you like.</li></ul><p>Following installation instruction it present below to install CUDA on WSL2.</p><p>CUDA location can be found at /usr/local/cuda-12.3 .</p><h4>Verify CUDA</h4><p>To see if CUDA is installed, we can run nvcc -V. nvcc is located at /usr/local/cuda-12.3/bin . Running the command will prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/534/1*Mn7JHnCpoTj5T7rpscYOZw.jpeg" /></figure><p>Not found? huh…?</p><p>That is because current bash shell’s <strong>PATH </strong>variable doesn’t included nvcc . Why don’t we include <strong><em>bin</em></strong> folder under <strong><em>cuda-12.3</em></strong> to the <strong>PATH</strong>.</p><p>To include it to <strong>PATH </strong>we can run</p><pre>export PATH=/usr/local/cuda-12.3/bin/:$PATH</pre><p>Let’s run nvcc -V again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*kOos5guUB0qBzxPktKWZdg.jpeg" /></figure><blockquote>The PATH variable is just a special variable that contains all of the directories that are automatically searched when we try to call a program.</blockquote><p><strong><em>One thing be notice here</em></strong></p><pre>export PATH=/usr/local/cuda-12.3/bin/:$PATH</pre><p>is <strong><em>temporarily </em></strong>and this setting will disappear once you exit Ubuntu. In order to <strong><em>make it permanent</em></strong>, we can write it to <strong><em>.bashrc</em></strong> file. Run</p><pre>vim ~/.bashrc</pre><p>and press i on keyboard to edit. add this</p><pre> export PATH=/usr/local/cuda-12.3/bin/:$PATH</pre><p>at end of <strong><em>.bashrc</em></strong> file. Press ESC on keyboard exit edit mode and press shift + : on keyboard then type wq and press Enter.</p><p>After it was written into <strong><em>.bashrc,</em></strong> you need to reload <strong><em>.bashrc</em></strong></p><pre>source ~/.bashrc</pre><blockquote>. bashrc (short for bash read command) is a configuration file for the Bash shell environment. Every time an interactive Bash shell session starts, the . bashrc script file executes.</blockquote><h4>Install cuDNN</h4><p>For cuDNN. We will install version 8.9.</p><blockquote>The <a href="https://developer.nvidia.com/cudnn"><strong>NVIDIA CUDA</strong></a>®<a href="https://developer.nvidia.com/cudnn"><strong> Deep Neural Network (cuDNN)</strong></a> is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.</blockquote><p>The first instinct is to go to official cuDNN website and download it then follow installing instruction. <strong><em>This will not work at least in my case</em></strong>.</p><p>To properly install cuDNN, you need to download it from <a href="https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/"><strong><em>here</em></strong></a><strong><em> </em></strong>or<strong><em> </em></strong>at cuDNN download page scroll down to bottom at Resource section click <a href="https://developer.download.nvidia.com/compute/cudnn/redist"><strong><em>Tarball and Zip Archive Deliverables</em></strong></a>. Then <a href="https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/"><strong><em>cudnn/</em></strong></a> follow by <a href="https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/"><strong><em>linux-x86_64/</em></strong></a>.</p><p>Find the one with name <a href="https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/cudnn-linux-x86_64-8.9.0.131_cuda12-archive.tar.xz"><strong><em>cudnn-linux-x86_64–8.9.0.131_cuda12-archive.tar.xz</em></strong></a> and download <strong><em>tar </em></strong>file with</p><pre>wget https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/linux-x86_64/cudnn-linux-x86_64-8.9.0.131_cuda12-archive.tar.xz</pre><p>Next run</p><pre>tar -xvf cudnn-linux-x86_64-8.9.0.131_cuda12-archive.tar.xz</pre><p>to extract file at currently working directory.</p><p>Copy every <strong><em>header files</em></strong> inside <strong><em>include folder</em></strong> under extracted cuDNN folder to /usr/local/cuda-12.3/targets/x86_64-linux/include/ by running this command</p><pre>sudo cp cudnn-linux-x86_64-8.9.0.131_cuda12-archive/include/cudnn*.h /usr/local/cuda-12.3/targets/x86_64-linux/include/</pre><p>Also copy every <strong><em>library files</em></strong> inside <strong><em>lib folder</em></strong> under extracted cuDNN folder to /usr/local/cuda-12.3/targets/x86_64-linux/lib/ by running this command</p><pre>sudo cp cudnn-linux-x86_64-8.9.0.131_cuda12-archive/lib/libcudnn* /usr/local/cuda-12.3/targets/x86_64-linux/lib/</pre><p>Change privilege for header and library files</p><pre>sudo chmod a+r /usr/local/cuda-12.3/targets/x86_64-linux/include/cudnn* /usr/local/cuda-12.3/targets/x86_64-linux/lib/libcudnn*</pre><blockquote>If you want to know chmod a+r means see <a href="https://en.wikipedia.org/wiki/Chmod"><strong><em>here</em></strong></a>.</blockquote><h4>The missing piece in puzzle</h4><p>At this point, it seems ready to rock. Run</p><pre>python3 -c &quot;import tensorflow as tf; print(tf.config.list_physical_devices(&#39;GPU&#39;))&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4uZMaX81UOM8Donc_84rKQ.jpeg" /></figure><p>If Tensorflow detect GPU then it will show up in list at last line. Unfortunately nothing show up. At last few lines I notice message <strong><em>Cannot dlopen some GPU libraries</em></strong> and <strong><em>Skipping registering GPU devices</em></strong>. What libraries?</p><p>The one library is <strong><em>lib64</em></strong> which is in inside CUDA folder and located at /usr/local/cuda-12.3/.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/815/1*09OiJrkdRUL-iZIvCdxK8g.jpeg" /></figure><p>To fix this issue, we run</p><pre>export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64:$LD_LIBRARY_PATH</pre><p>which add <strong><em>lib64</em></strong> to <strong>LD_LIBRARY_PATH</strong> variable.</p><blockquote>LD_LIBRARY_PATH tells the dynamic link loader (ld. so — this little program that starts all your applications) where to search for the dynamic shared libraries an application was linked against. Multiple directories can be listed, separated by a colon (:), and this list is then searched <em>before</em> the compiled-in search path(s), and the standard locations (typically /lib, /usr/lib, …).</blockquote><p><strong><em>One thing be notice here</em></strong></p><pre>export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64:$LD_LIBRARY_PATH</pre><p>is <strong><em>temporarily </em></strong>and this setting will disappear once you exit Ubuntu. In order to <strong><em>make it permanent</em></strong>, we can write it to <strong><em>.bashrc</em></strong> file. Run</p><pre>vim ~/.bashrc</pre><p>and press i on keyboard to edit. add this</p><pre>export LD_LIBRARY_PATH=/usr/local/cuda-12.3/lib64:$LD_LIBRARY_PATH</pre><p>at end of <strong><em>.bashrc</em></strong> file. Press ESC on keyboard exit edit mode and press shift + : on keyboard then type wq and press Enter.</p><p>After it was written into <strong><em>.bashrc,</em></strong> you need to reload <strong><em>.bashrc</em></strong></p><pre>source ~/.bashrc</pre><p>Let’s run</p><pre>python3 -c &quot;import tensorflow as tf; print(tf.config.list_physical_devices(&#39;GPU&#39;))&quot;</pre><p>again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-V3TChN5SLE1nE-qvVNYeg.jpeg" /></figure><p>Finally Tenosrflow is able to detect my GPU.</p><h3>WSL2 import &amp; export &amp; unregister</h3><p>This is not part of this walkthrough but I use this 2 commands during experiment. You can tell WSL2 to export current Ubuntu to tar file at a location on Windows and then import it later. With these features you can snapshot your Ubuntu and restore it.</p><p>Export your current Ubuntu as a distribution with name Ubuntu.</p><pre>wsl --export Ubuntu &quot;export to directory incldue filename with tar extension&quot;</pre><p>Unregister/Uninstall current Ubuntu.</p><pre>wsl --shutdown<br>wsl --unregister Ubuntu</pre><p>Import distribution.</p><pre>wsl --import Ubuntu &quot;import to directory&quot; &quot;tar filename&quot;</pre><p>Finally you can set username as default user for imported distribution. Username that you typed during first time WSL2 Ubuntu installing.</p><pre>Ubuntu config --default-user [username]</pre><h3>PyTorch is easier</h3><p>With <a href="https://pytorch.org/get-started/locally/">PyTorch</a> , all I have to do is</p><pre>pip3 install torch torchvision torchaudio</pre><p>Let’s verify it if GPU is working.</p><pre>python3 -c &quot;import torch;print(torch.cuda.is_available())&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3memHYsxGdDGZUaArtHKQw.jpeg" /></figure><p>I don’t even need those tedious works in Tensorflow.</p><blockquote>The PyTorch binaries ship with all CUDA runtime dependencies and you don’t need to locally install a CUDA toolkit or cuDNN</blockquote><h3>Conclusion</h3><p>There is quite a bit of works to do in order to setup Tensorflow with GPU for WSL2. At this time of writing this walkthrough, official installation <a href="https://www.tensorflow.org/install/pip#windows-wsl2"><strong><em>here</em></strong></a><strong><em> </em></strong>did not<strong><em> </em></strong>explain clearly. Even we fix the issue, those tedious works is very painful for just to enable Tensorflow to detect GPU.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f06651df25b7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/tensorflow-with-wsl-f06651df25b7">Tensorflow with WSL2</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 1 — Setup — Linux Environment]]></title>
            <link>https://medium.com/software-dev-explore/part-1-setup-linux-environment-dd5a56659fac?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd5a56659fac</guid>
            <category><![CDATA[virtualbox]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[ubuntu]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Sun, 14 Apr 2024 02:39:00 GMT</pubDate>
            <atom:updated>2024-04-14T02:39:00.294Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 1 — Setup — Linux Environment</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*a-p5K2yNLtt3v36f" /><figcaption>Photo by <a href="https://unsplash.com/@thomasbennie?utm_source=medium&amp;utm_medium=referral">Thomas Bennie</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Distributions</h3><p>Linux is an <strong>Operating System</strong> made from a collection of software that are based on <strong>Linux kernel</strong>.</p><p>Linux comprise:</p><ul><li><a href="https://www.kernel.org/">Linux kernel</a></li><li><a href="https://www.gnu.org/home.en.html">GNU </a>shell utilities</li><li>Graphical desktop environment</li><li>More</li></ul><p><a href="https://distrowatch.com/">Here </a>we can checkout all Linux distributions.</p><h3>Install Ubuntu</h3><p>We will use VirtualBox a virtual machine to run Ubuntu.</p><h4>Install VirtualBox</h4><p>Download <a href="https://www.virtualbox.org/wiki/Downloads">VirtualBox</a> for your operating system and install it.</p><p>It is also recommend to install <strong>VirtualBox Extension Pack</strong>.</p><h4>Install Ubuntu</h4><p>Download <a href="https://ubuntu.com/download/desktop">Ubuntu</a>. And start VirtualBox.</p><p>Create a new Virtual Machine. <em>Machine -&gt; New</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*IcEwpB6Yig_KQw09asFxmQ.jpeg" /></figure><p>Fill the fields in the prompt window and click next.</p><ul><li><strong>Name</strong>: name of your virtual machine</li><li><strong>Folder</strong>: location where your virtual machine to be stored</li><li><strong>ISO Image</strong>: the ISO image you downloaded for Linux. In this case is Ubuntu ISO image</li><li><strong>Type</strong>: select Linux</li><li><strong>Version</strong>: in this case Ubuntu (64-bit)</li></ul><p>Check <em>Skip unattended Installation</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*obtp02eOv25oP5ozBi5yOg.jpeg" /></figure><p>Configure virtual machine hardware.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*arLTvLYHYx2yB7-Mi4gvYA.jpeg" /></figure><p>Configure virtual machine hard disk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*geIwhriBC_8hEZUzBc3BHw.jpeg" /></figure><p>Confirm your final summary and click Finish.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*ssBxRoGaxsU0bJcsdp0Wfg.jpeg" /></figure><p>Select the virtual machine you just created and start it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*fjyphiQg42OeMWUyIC-Ifw.jpeg" /></figure><p>Select <em>Try or Install Ubuntu</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*YvBa2b-esvZ6PgjBg9ur-g.jpeg" /></figure><p>Select your language and click <em>Install Ubuntu</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jepub7FhpmQRv-8MOrgk9w.jpeg" /></figure><p>Configure your keyboard layout.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u9nx48Ipc5GP61KFnC56vA.jpeg" /></figure><p>Make sure you have internet connection and check two checkbox under <em>Other options</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eszuGwk6W0dtGHpH0KQjcA.jpeg" /></figure><p><strong>If you use virtual machine then leave it as default otherwise you can select Something else if you are on main operating system</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W9ogDIhOnaQlOm2axISS7g.jpeg" /></figure><p>If a window prompt click Continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cXPPlRh3JDdOVDGicUUOGA.jpeg" /></figure><p>Select your location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g5A41IQeRj1LGxJToo_XHQ.jpeg" /></figure><p>Fill your name and password then Continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oyTga7txBgwLkqZoNCRhnQ.jpeg" /></figure><p>Wait for installation to finish.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYZS-aiEblw9MugqnTCVfg.jpeg" /></figure><p>Click Restart Now when installation completed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JAa9IJU0NGEOkPUP_vP3cg.jpeg" /></figure><p>Press Enter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vu47eKcpAthkv2-m54GEgw.jpeg" /></figure><p>Login with your user and password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NyKUfju4Q89fvbg5RGKmCw.jpeg" /></figure><p>We have Ubuntu opertating system installed successfully. 🎉</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nH73-npHxXGVQwema2TPGQ.jpeg" /></figure><h3>After install Ubuntu</h3><h4>Update packages</h4><p>Right click on desktop and select Open in Terminal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zte6No9U1-ChCOUPmMG3AA.jpeg" /></figure><p>In terminal type command sudo apt update . Enter password and hit Enter on keyboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/993/1*5X4zx7XxgPHJxO3yDwdlLw.jpeg" /></figure><p>Then run command sudo apt full-upgrade</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*cyiLxZPVQ650UMAtZS5c4Q.jpeg" /></figure><h4>Install Guest Additions</h4><p>For better support we can install Guest Additions come with Virtual Box.</p><p>Insert the Guest Additions CD Image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Bw7u7pjyY-sDwymDiIKXg.jpeg" /></figure><p>Type command cd /media/user/VBox_GAs_7.0.12/ hit Enter and then ls .</p><blockquote>user in the path is your user name.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*nw8KB-RLxDzZzuA3m0J2RA.jpeg" /></figure><p>To install it, we run command sudo bash VBoxLinuxAdditions.run . Wait for it to finish installation and run command reboot to restart virtual machine.</p><h4>Snapshot</h4><p>Virtual Box allow you to snapshot current state of virtual machine in case you mess up the operating system and want to restore your virtual machine to current state.</p><p>To create snapshot. First power down virtual machine then in Virtual Box Manager screen select the virtual machine on left side and click Take button at top.</p><p>To restore. In Virtual Box Manager screen select the virtual machine and then on right side select snapshot you want to restore to. And click Restore button at top.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd5a56659fac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/part-1-setup-linux-environment-dd5a56659fac">Part 1 — Setup — Linux Environment</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Part 1 — Tensor]]></title>
            <link>https://medium.com/software-dev-explore/part-1-tensor-1b6cc9b68b73?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b6cc9b68b73</guid>
            <category><![CDATA[pytorch]]></category>
            <category><![CDATA[tensor]]></category>
            <category><![CDATA[numpy]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Sat, 13 Apr 2024 09:02:04 GMT</pubDate>
            <atom:updated>2024-04-13T09:02:04.480Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 1 — Tensor</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iJIITngIb7jfwe60" /><figcaption>Photo by <a href="https://unsplash.com/@rush_intime?utm_source=medium&amp;utm_medium=referral">rashid khreiss</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>This is a basic knowledges about tensor creation, reshape and copy in PyTorch. In addition, see how can we compute tensor on both CPU and GPU. Last use Tensor and Numpy interchangeably.</p><h3>Tensor manipulation</h3><h4>Tensor</h4><p>In general a <strong>Tensor</strong> is an array contain three or more dimensions.</p><p>With PyTorch we are able to create tensor easily.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e51b0766c34e44d053bd07ecd2786aab/href">https://medium.com/media/e51b0766c34e44d053bd07ecd2786aab/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*8q-F13Ir9KzBeK8jDRLkJA.png" /></figure><p>We are able to check its size or shape.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/65cf1d79e010639b527ff6d64de35abc/href">https://medium.com/media/65cf1d79e010639b527ff6d64de35abc/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*_K0Mb6etHVCBag2EO9QgTw.png" /></figure><ul><li><strong>scalar</strong>: Is a tensor with a single value and it is 0 dimension.</li><li><strong>vector</strong>: Is a tensor and it is 1 dimension(1D).</li><li><strong>matrix</strong>: Is a tensor and it is 2 dimension(2D).</li><li><strong>tensor</strong>: Is a tensor and it is 3 dimension(3D).</li></ul><p>They are all type of tensor but with different dimension. To interpret a given shape of a tensor, we can take an example torch.Size([2, 3, 4]) this means in this tensor it has 2 elements in first dimension, 3 elements in second dimension and 4 elements in third dimension, therefore we know the number of dimension is 3 and we also know how many elements in each dimensions. Notice scalar has no dimension.</p><h4>Reshape</h4><p>We can reshape a tensor with PyTorch. And there are 2 methods we can use.</p><ul><li>view(): Return a tensor with desired shape and share underlying data with the original tensor. In other word it does not create a new tensor.</li><li>reshape(): May or may not create a copy.</li></ul><p>The recommend way to reshape a tensor is to use view().</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d25edea14499ecfff1b2c175facca8f2/href">https://medium.com/media/d25edea14499ecfff1b2c175facca8f2/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*vhpStyKiOo1hLeuKlMsqzA.png" /></figure><blockquote>matrix here is a tensor with 1. for all values. And it is from previous section.</blockquote><p>We can see that <em>matrix</em> is a 2x3 tensor and it had been reshaped to 1x6 tensor. We change its value to <em>2.</em> at [0,1] first row and second element in 1x6 tensor and this result in original <em>matrix</em> been changed. <strong><em>They are shared the same underlying data</em></strong>.</p><h4>Copy</h4><p>2 ways to copy a tensor.</p><ul><li>new_tensor()</li><li>clone()</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/131f6d2122ec43f48d3da38f699b8867/href">https://medium.com/media/131f6d2122ec43f48d3da38f699b8867/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*CmXVfB9vcuQXg_wnOSiNeg.png" /></figure><p>We can see that change in new tensor do not change original tensor.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fad951c054ab654f668c1b047c2e4650/href">https://medium.com/media/fad951c054ab654f668c1b047c2e4650/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*BV0y5XnxmkNjZ-1hY4dYEQ.png" /></figure><p>We are doing the same thing again except this time we use clone() and detach().</p><blockquote>detach() tell PyTorch to remove this tensor from computation graph.</blockquote><h3>GPU, CPU CUDA</h3><p>If your computer has NVIDA dedicate graphics card you might be able to put tensor on to GPU for computation. All tensors will be on CPU for computation if you don’t have GPU available.</p><p><strong><em>CUDA</em></strong></p><blockquote>Compute Unified Device Architecture is a parallel computing platform and application programming interface that allows software to use certain types of graphics processing units for accelerated general-purpose processing, an approach called general-purpose computing on GPUs</blockquote><p>Parallel computing allow tensor to be computed faster than CPU.</p><h4>Convert Numpy to Tensor</h4><p>Numpy is used very often when doing machine learning or deep learning therefore to convert Numpy data to tensor back and forth is important.</p><p>2 ways to convet Numpy to tenosr</p><ul><li>as_tensor() : Share underlying data with original Numpy array.</li><li>from_numpy() : Share underlying data with original Numpy array.</li></ul><p>To convert tensor back to Numpy</p><ul><li>numpy() : Convert back to Numpy from tensor.</li></ul><h4>CPU</h4><p>Following code will only work on CPU.</p><p>Tensor changed when original Numpy changed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de86413de364da88b33ccfac53da3080/href">https://medium.com/media/de86413de364da88b33ccfac53da3080/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*iJM_Juh12DpWfKTYCdKwoA.png" /></figure><p>Tensor will not change even we change original Numpy.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e05dbcc91bf60b05c698fda2d7c66029/href">https://medium.com/media/e05dbcc91bf60b05c698fda2d7c66029/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*4rLT-03aTJGfrPnnFC8n-Q.png" /></figure><p>To change tensor data type. It change from float64 to float32.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/888a78fa3317011769e1debb3e50cee1/href">https://medium.com/media/888a78fa3317011769e1debb3e50cee1/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/288/1*k3cZGmdGRgvoXNNyctYJpg.png" /></figure><p>To convert back to Numpy from tensor.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de0d4b037365d7470c65b04583e2d39d/href">https://medium.com/media/de0d4b037365d7470c65b04583e2d39d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*LZ9WnljB-0jOZ4hS7bopdQ.png" /></figure><h4>GPU</h4><p>The way to put tensor on to GPU is to use to().</p><blockquote>Always use to() during tensor creation in order to put tensor on proper device either CPU or GPU. This way will make sure there is no issue when sharing your code with others.</blockquote><p>Let’s setup device and see device information</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/065865e5562cb89a7cdf72c4062c46c2/href">https://medium.com/media/065865e5562cb89a7cdf72c4062c46c2/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/464/1*LMA5LKuf3dax55m0ZEsdfw.png" /></figure><p>Create a tensor from Numpy and put it on GPU.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5b934940382236f1740ded53ba3e6c6/href">https://medium.com/media/b5b934940382236f1740ded53ba3e6c6/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/228/1*E5iIShaWfeBziS9KH_PU-w.png" /></figure><p>From GPU to Numpy. <strong><em>Remember Numpy only work on CPU not GPU</em></strong>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1d159517c46f9aea63cad43af9d85b10/href">https://medium.com/media/1d159517c46f9aea63cad43af9d85b10/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/246/1*3qvS3OHytsUp_d_u1wmx4Q.png" /></figure><blockquote>Always use cpu() first then numpy() even there is no GPU available. This way will make sure there is no issue when sharing your code with others.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b6cc9b68b73" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/part-1-tensor-1b6cc9b68b73">Part 1 — Tensor</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gradient Descent]]></title>
            <link>https://medium.com/software-dev-explore/gradient-descent-9003193b4741?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/9003193b4741</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Fri, 22 Mar 2024 05:18:24 GMT</pubDate>
            <atom:updated>2024-03-23T02:18:24.315Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*l3O29f49mbm7SH7z" /><figcaption>Photo by <a href="https://unsplash.com/@kartikvastava?utm_source=medium&amp;utm_medium=referral">Kartikeya Srivastava</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is it ?</h3><p><strong>Gradient Descent</strong> is an <strong>iterative technique</strong> that is used in machine learning and deep learning to find the <strong>best possible</strong> set of parameters or coefficients for a model, data points and loss function. In other words, it is an iterative method to optimize a model’s parameters.</p><p>Gradient Descent technique is employed in <a href="https://en.wikipedia.org/wiki/Backpropagation"><strong><em>backpropagation </em></strong></a>stage during model training. And model learn from data and getting better and better is because of it.</p><h3>Parameters ?</h3><p>One might ask what parameters in a model? To illustrate parameters in a model, I will use a simple model namely <strong>Linear Regression</strong>. A Linear Regression model would look like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*twyQA0sgvTmboFA_ZQ7G5A.png" /><figcaption>formula for a straight line</figcaption></figure><ul><li>w: This is a <strong>parameter</strong>. A weight or slop. Tell us how much y increase on average when x increase by one unit</li><li>b: This is a <strong>parameter</strong>. A bias or intercept. Tell us expected average for y when x is zero</li><li>x: This is a <strong>feature </strong>or data point that used to predict y</li><li>y: This is <strong>label </strong>or prediction</li></ul><p>An example of salary prediction with this Linear Regression model. We define.</p><ul><li>w: Wage increase per years of experience. Says 500</li><li>b: Minimum wage. Says 1000</li><li>x: Years of experience</li><li>y: Predicted salary</li></ul><p>Then we can see.</p><p>Salary = wage increase per years of experience * years of experience + minimum wage</p><p>If we have no experience then our salary would be y = 500 * 0 + 1000 where y = 1000 as result. We can say when you have no experience(x) then your salary(y) is 1000 hence y = b.</p><ul><li>For 1 year experience: y = 500 * 1 + 1000 where y = 1500</li><li>For 2 year experience: y = 500 * 2 + 1000 where y = 2000</li><li>For 3 year experience: y = 500 * 3 + 1000 where y = 2500</li></ul><p>The pattern here is y increase 500 when x increase by 1 unit. We can say when you have experience(x) increase by 1 year then your salary(y) increase by 500.</p><p>Remember there are 2 parameters both w and b. Now you can see <strong>Gradient Descent</strong> is an iterative method used to find out what is optimized values for w and b so prediction can be as accurate as possible.</p><h3>Error and Loss</h3><p>In order to optimize model’s parameters, we need to know how much difference(error) between predicted value and actual value. The loss would be calculated for a set of data.</p><p>An error would be.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*weIBH-IwjUnQfexNLVAW1Q.png" /></figure><ul><li>y(with angle bracket): Prediction value</li><li>y: Actual value(label or answer)</li></ul><blockquote>i here is a particular data point from 0 ~ N</blockquote><p>Error and loss is different.</p><ul><li>Error: The difference between <strong>actual value</strong> and <strong>predicted value</strong>.</li><li>Loss: <strong>Aggregation of errors</strong> on a set of data.</li></ul><p>For a Linear Regression model, it’s loss can be calculated with <strong>MES(Mean Square Error) </strong>Average of all square errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtjY5WgQBlyp79Nu2xvoFw.png" /></figure><ul><li>wx+b: Prediction value</li><li>y: Actual value</li></ul><p>Model will have accurate prediction when loss is low. In other words <strong>the lower the loss the better the prediction</strong>. When loss is at it minimum, w and b parameters is at its best.</p><h3>Gradient</h3><p>How to know the change or loss when a or b parameters changed? Answer is gradient.</p><p><strong>Gradient tell us how much a value changed when we slightly vary other value</strong>. In our case how much loss will change when we vary wand b separately.</p><p>Gradient is also known <strong>partial derivative</strong>. Our Linear Regression model has 2 parameters a and b and partial derivate can help us to find out the gradient of wand gradient of b.</p><p>A gradient of a straight line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*7hVH-T3db-P5RytGsqJlwQ.jpeg" /><figcaption><a href="https://www.onlinemathlearning.com/gradient-graphs.html">https://www.onlinemathlearning.com/gradient-graphs.html</a></figcaption></figure><p>Imaging X axis is w or b parameters and Y axis is loss then we would have much clear picture of what is gradient.</p><p>Partial derivate of loss respect to w parameter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*SOMUMXNTsk3qykMGlStBeg.png" /></figure><p>Partial derivate of loss respect to b parameter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*GDC6DclmFK19lk8gHFbsWw.png" /></figure><h4>Partial Derivative</h4><p>We can use <a href="https://en.wikipedia.org/wiki/Chain_rule">chain rule</a> to calculate partial derivate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*VvI2XrKsKddME8MIE423PA.png" /></figure><p>Calculate from outer function to inner function and multiply them together.</p><p>Some knowledges about derivative.</p><p>The derivative with respect to x(variable).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*PmERz3HJl05aSZ3vVLootg.png" /></figure><p>so we can see an example below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*C63V4SVO9T4LSotHC1bC1w.png" /></figure><p>The derivative with respect to c(constant) is always 0.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*3v9cEgN4XMzmms7bPGclCQ.png" /></figure><p>With this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*oafWYSbPi7V5XRqRHnFNKg.png" /></figure><p>Now we can do partial derivate with respect to w and b.</p><h4>Partial derivate with respect to w</h4><p>Keep expression inside not change and derivative of ()² is 2() .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*amm0qio_ghMLFzRzpMB6mQ.png" /></figure><p>Next for expression in side. First ignore 2()treat b and y as constant. Because derivative of constant is 0 hence wx + 0-0 is wx. x stay in place since it is with w. Derivative of w is 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*Dvr_dlQC1X7O4yVzUDPdyA.png" /></figure><p>Multiply them together.</p><blockquote>wx+b is equal to y(hat)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WN75x1k7VBbSyFHaFVQr-g.png" /></figure><p>Finally we got.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*SOMUMXNTsk3qykMGlStBeg.png" /></figure><h4>Partial derivate with respect to b</h4><p>Keep expression inside not change and derivative of ()² is 2() .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*amm0qio_ghMLFzRzpMB6mQ.png" /></figure><p>Next for expression inside. First ignore 2(). Derivative of b is 1 and wx and y are 0.</p><blockquote>w here become constant value</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/530/1*kD3nVweYKtPR_aqwuW5lWQ.png" /></figure><p>Multiply them together.</p><blockquote><em>wx+b</em> is equal to y(hat)</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c5UiopcdcSh_0xXEw5GPNQ.png" /></figure><p>Finally we got.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*GDC6DclmFK19lk8gHFbsWw.png" /></figure><p>Partial derivative(Gradient) with respect to each w and b in graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ww7Mpw_nxMfqgPbMMkK4FQ.jpeg" /></figure><h3>Update parameters</h3><p>Now we know loss and know how to find gradient for w and b. In each iteration of Gradient Descent we need to update both parameters w and b. However there is another parameter namely <strong>Learning Rate</strong> involved.</p><p>Learning Rate is a parameter that can be adjusted manually therefore it is also known as hyper-parameter.</p><p>Gradient Descent in graph look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*xTaI_tClf1z80lcWCCZXVw.jpeg" /><figcaption><a href="https://blog.gopenai.com/understanding-of-gradient-descent-intuition-and-implementation-b1f98b3645ea">https://blog.gopenai.com/understanding-of-gradient-descent-intuition-and-implementation-b1f98b3645ea</a></figcaption></figure><blockquote>Cost here is another term for loss</blockquote><p>In the beginning parameters are initialized randomly and then it look like ball start rolling down hill step by step until it reach bottom of hill where the loss is at minimum.</p><p>Steps or <em>Learning step</em> in graph is our <strong>Learning Rate</strong>. High learning rate would overshoot and end up in another side of hill and too small would take forever to reach bottom of hill.</p><p>In Gradient Descent, there is only one learning rate but a model can have many parameters. Thus choosing proper <strong>Learning Rate</strong> is another important topic.</p><p>To update parameter w and b.</p><p><strong>New parameter value = previous parameter value — learning rate * gradient</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*avMJIZIXZGplgO9m1R2cHg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*hDn8T3_KwiYiN6liRRJKtA.png" /></figure><p>Eta(learning rate).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/40/1*QSmALuNSer1f566keAL1fg.png" /></figure><h3>Code</h3><p>The complete code is <a href="https://colab.research.google.com/drive/1jWAEYlWG02SRS0v6u4cAxblCkFWney3C?usp=sharing"><strong><em>here</em></strong></a>.</p><p>To code Gradient Descent is pretty straightforward.</p><p>First we need to generate data points then do Gradient Descent.</p><p>We can define number of iterations for Gradient Descent. And each iterations we will do the following.</p><ul><li>Forward pass(Make prediction)</li><li>Compute loss</li><li>Compute gradient</li><li>Update parameters</li></ul><h4>Generate data</h4><p>The purpose of <em>noise </em>in the code is to spread data point a bit otherwise data points will become a straight diagonal line.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92816cb60e34d962b8b03e821ef8ac73/href">https://medium.com/media/92816cb60e34d962b8b03e821ef8ac73/href</a></iframe><p>Let’s see what data look like and here we also plot a regression line to represent w and b parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f8c9777013168db6a6b4626aa25f8b7/href">https://medium.com/media/2f8c9777013168db6a6b4626aa25f8b7/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*shFwpTMA0838l7yU38xUKQ.jpeg" /></figure><p>The red line is what we expected to be after Gradient Descent.</p><h4>Gradient Descent</h4><p>p_w and p_b are initialized randomly and we expect them to be updated in each Gradient Descent iterations and move toward w and b parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1b88bc5fa29a53247cbcc0ce5cd08728/href">https://medium.com/media/1b88bc5fa29a53247cbcc0ce5cd08728/href</a></iframe><p>Print the result to see p_w and p_b parameters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1dff30d2c2782b1340777c1055a0380/href">https://medium.com/media/b1dff30d2c2782b1340777c1055a0380/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/236/1*c_097EiJql7ZgxMy3m81Bw.jpeg" /></figure><p>We see p_w and p_b are very close to w and b .</p><p>We can visualize the regression line change in motion</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff49439d22b97abc7b37c51441ff33e6/href">https://medium.com/media/ff49439d22b97abc7b37c51441ff33e6/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*mZ2L3E0RzEdInKhhmSsXag.gif" /></figure><p>Green dash line moving closer and closer to red solid line each epochs(Iteration).</p><p>Here we also can visualize w and b parameters approaching to their optimal value.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0718b4b20250c3509c79d98da1bc7304/href">https://medium.com/media/0718b4b20250c3509c79d98da1bc7304/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*XEx14fnBAKw2H7bZZr5egg.gif" /></figure><p>The color represent loss value and the darker the color the higher the loss. The purple line represent w and b parameters.</p><h3>Conclusion</h3><p>Gradient Descent is a technique to find best possible set of parameters for a model. In our case Linear Regression model, we have only 2 parameters w and b then we use Gradient Descent technique to optimize w and b.</p><p>For a more complex model, it literally doing the same thing except for more than 2 parameters. A complex model may have thousand or million parameters.</p><p>Finally, we have a clear picture of what Gradient Descent is.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9003193b4741" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/gradient-descent-9003193b4741">Gradient Descent</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python import module from root directory?]]></title>
            <link>https://medium.com/software-dev-explore/python-import-module-from-root-directory-1a7b36892ab7?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/1a7b36892ab7</guid>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Mon, 12 Feb 2024 09:21:48 GMT</pubDate>
            <atom:updated>2024-02-12T09:42:24.089Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DE2dNRpJhfxQ659H" /><figcaption>Photo by <a href="https://unsplash.com/@hooverpaul55?utm_source=medium&amp;utm_medium=referral">Paul Teysen</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>Import module in Python is fairly easy, here is example.</p><pre>import my_module as module<br>from my_module import say_hello</pre><p>However it will throw an error in certain case.</p><p>Recently, I encounter this error during development. In a project I have few services that run in <a href="https://www.docker.com/"><em>docker</em></a>, which is a containerized light weight virtual machine. Those services may have common functionalities therefore I make them as a Python module but the example above for importing Python module is no longer work.</p><h3>The problem</h3><pre>python_modules_example/<br>┣ app/<br>┃ ┣ app_modules/<br>┃ ┃ ┣ utils.py<br>┃ ┃ ┗ __init__.py<br>┃ ┗ app.py<br>┣ common_modules/<br>┃ ┣ utils.py<br>┃ ┗ __init__.py<br>┣ .gitignore<br>┗ main.py</pre><blockquote>The __init__.py files are required to make Python treat directories containing the file as packages. See <a href="https://docs.python.org/3/tutorial/modules.html#packages">here</a>.</blockquote><p>A project file and folder structure above. There is no problem to import modules from <em>common_moduls </em>and <em>app_modules </em>within <em>main.py</em>.</p><p><em>main.py</em></p><pre>import common_modules.utils as common_utils<br>import app.app_modules.utils as app_utils<br><br>if __name__ == &quot;__main__&quot;:<br>    common_utils.say_hello()<br>    app_utils.say_hello()</pre><p>But this is no longer the case for <em>app.py</em>.</p><p><em>app.py</em></p><pre># import module from outside of app folder<br>import common_modules.utils as common_utils<br><br># import module from inside app folder<br>import app_modules.utils as app_utils<br><br>if __name__ == &quot;__main__&quot;:<br>    common_utils.say_hello()<br>    app_utils.say_hello()</pre><p>If I run <em>app.py</em> then an error occur.</p><pre>Traceback (most recent call last):<br>  File &quot;./app/app.py&quot;, line 2, in &lt;module&gt;<br>    import common_modules.utils as common_utils<br>ModuleNotFoundError: No module named &#39;common_modules&#39;</pre><p>The error says <em>common_modules</em> not found. Take a look at directory path of <em>common_modules</em>, it is under root directory.</p><h3>Why</h3><p>The reason it throw an error says module not found is because <em>common_modules </em>is not at same or in subdirectory of <em>app.py</em>.</p><p>When we run <em>app.py</em>, Python will search for modules at same directory or in subdirectory of <em>app.py</em> and any directory other than that will not be included. That explained why <em>main.py</em> work fine but not work for <em>app.py</em>.</p><p>If we comment out all code in <em>app.py</em> and add these two lines.</p><pre>import sys<br>print(sys.path)</pre><p>It will print a list of module search paths as string in my local machine.</p><pre>[<br>&#39;C:\\Users\\tomne\\Desktop\\python_modules_example\\app&#39;, <br>&#39;C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\python38.zip&#39;, <br>&#39;C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\DLLs&#39;, <br>&#39;C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\lib&#39;, <br>&#39;C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38&#39;, <br>&#39;C:\\Users\\tomne\\AppData\\Local\\Programs\\Python\\Python38\\lib\\site-packages&#39;<br>]</pre><blockquote>A module search path is initialized when Python starts. This module search path may be accessed at <a href="https://docs.python.org/3/library/sys.html#sys.path">sys.path</a>.</blockquote><blockquote>The <a href="https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH">PYTHONPATH</a> environment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path.</blockquote><blockquote>see <a href="https://docs.python.org/3/library/sys_path_init.html">here</a>.</blockquote><p>As you can see app directory is at first in list but there is no sight of <em>common_modules </em>directory.</p><h3>Solution</h3><p>The solution is simple we can add root directory to the list so Python will search entire directory.</p><p><em>app.py</em></p><pre>import os<br>import sys<br># insert root directory into python module search path<br>sys.path.insert(1, os.getcwd())<br><br># import module from outside of app folder<br>import common_modules.utils as common_utils<br><br># import module from inside app folder<br>import app_modules.utils as app_utils<br><br>if __name__ == &quot;__main__&quot;:<br>    common_utils.say_hello()<br>    app_utils.say_hello()</pre><p>We insert the root directory into the list with one line of code before we importing any modules.</p><pre>sys.path.insert(1, os.getcwd())</pre><blockquote>os.getcwd() return a path to current working directory in string</blockquote><p>Now when we run <em>app.py</em> then it work as we expected without issue.</p><p><a href="https://github.com/tomneo2004/python_import_modules_example"><em>Here</em></a><em> </em>is the example project.</p><h3>Conclusion</h3><p>Often we do not need to do this. However in some cases we might need to import a module from different directory than where we run our Python script. And this solution solve the problem we encounter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1a7b36892ab7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/python-import-module-from-root-directory-1a7b36892ab7">Python import module from root directory?</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Neural Style Transfer VGG19]]></title>
            <link>https://medium.com/software-dev-explore/neural-style-transfer-vgg19-dab643ec6160?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/dab643ec6160</guid>
            <category><![CDATA[vgg19]]></category>
            <category><![CDATA[neural-style-transfer]]></category>
            <category><![CDATA[pytorch]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Tue, 16 Jan 2024 08:38:05 GMT</pubDate>
            <atom:updated>2024-05-26T02:13:34.818Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tiO1oiJiAtldv4Q_" /><figcaption>Photo by <a href="https://unsplash.com/@alinnnaaaa?utm_source=medium&amp;utm_medium=referral">Alina Grubnyak</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>There are 2 techniques appear in horizon when it comes to styling an image. They are both <strong>GAN</strong> and <strong>Neural Style Transfer</strong>. GAN stand for <a href="https://en.wikipedia.org/wiki/Generative_adversarial_network"><em>Generative Adversarial Network</em></a>, which is a generative model and use 2 networks compete each other to achieve style transfer outcome. <a href="https://en.wikipedia.org/wiki/Neural_style_transfer"><em>Neural Style Transfer</em></a> on the other hand use pre-trained model and algorithm to produce desired result.</p><p>Both approaches are quite different but they are in the same domain. GAN is modern approach to generate styled image and it is much faster but I am going to take look at Neural Style Transfer and learn how it works under the hood.</p><p>There is an <a href="https://arxiv.org/abs/1508.06576"><em>paper </em></a>about Neural Style Transfer.</p><h3>Objective</h3><p>The objective here is given 2 images. One image is a content (original) image and an other image is style image then apply style image on content image. The final produced image should have content image with particular style from style image.</p><blockquote>Style image here refer to an image painted by painter(Picasso, Van Gogh…) with certain unique style whereas content image refer to any kind of image</blockquote><p>I will use a painting <a href="https://en.wikipedia.org/wiki/The_Scream"><strong><em>The Scream</em></strong></a> by Edvard Munch for the style image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zi8POdCW2m8ChL6eZ-qhjw.png" /><figcaption>Style image</figcaption></figure><p>For Content image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*eB9bWW1N6CRr5_Bx_FcqnA.jpeg" /><figcaption>Content image</figcaption></figure><p>Final styled image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*TwkmZD4AL18598SWSuk1yw.jpeg" /><figcaption>Content image with style</figcaption></figure><p>This is final produced image with styled and this is my primary objective.</p><h3>VGG19</h3><p>VGG19 is a computer vision model and capable of classify 1000 objects. The model had been trained with millions of images. All computer vision model are base on <a href="https://en.wikipedia.org/wiki/Convolutional_neural_network"><em>Convolutional Neural Network</em></a>(CNN) including VGG19.</p><h4>Convolutional Neural Network</h4><p><a href="https://en.wikipedia.org/wiki/Convolutional_neural_network"><em>Convolutional Neural Network</em></a><em> </em>is able to learn features(patterns) from image dataset in order to classify number of different image.</p><p>A generic CNN consist of an <em>Input layer</em>, <em>number of blocks</em>, <em>Fully-connected</em> <em>layer</em> and <em>Output layer</em>. Each blocks have 1 or more <em>Convolutional layer </em>and<em> ReLU layer, Pooling layer </em>is always at end of block.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*5_CVS3uY1dErTaCxYw_Kxw.png" /><figcaption>A typical Convolutional Neural Network</figcaption></figure><p>In the image <em>Convolution layer</em>, <em>ReLU layer</em> and <em>a Pooling layer</em> all together form as a block. Image get passed into the block and <em>fully connected layer</em> in order to classify image. <strong><em>Convolution layer</em></strong><em> </em>is the key in whole network because it can learn features from images. <a href="https://poloclub.github.io/cnn-explainer/"><strong><em>Here</em></strong></a> is a CNN explainer explained CNN in an interactive way.</p><ul><li>Convolution layer: a layer can learn features from image</li><li>ReLU layer: an activation layer. There are many other different activation layers but ReLU is a common one of all</li><li>Pooling layer: a layer to condense image into smaller size of image and maintain as much information as possible from original image.</li></ul><blockquote>Convolution layer is always follow by a ReLU layer and only 1 Pooling layer at end of block</blockquote><h4>VGG19 Network</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3z0zGneRIqkQyxy4-cUbfg.png" /><figcaption>VGG19</figcaption></figure><p>The different between VGG19 and a typical CNN is VGG19 include more blocks and in each blocks contain at least 2 Convolutional layer. In addition it has 3 Fully-connected layers.</p><p>From the image I can see 5 blocks(green + blue as 1 block) in entire network. The minimum Convolutional layer in each block is 2 for example the green square at most left with <em>conv1_1</em> and <em>conv1_2</em>. <em>conv1_1</em> the first number means block and second number means layer. For example conv4_3 means Convolutional layer 3 in block 4.</p><blockquote>ReLU layer is not shown in the image because ReLU layer is always after Convolutional Layer.</blockquote><p>Another thing is <em>depth</em>. <em>Depth</em> also known as feature maps so if depth is 64 then the Convolutional layer will produce 64 feature maps.</p><ul><li>Feature maps(Depths): similar to color channels in an image but it contain features(patterns) from an image. Each of feature maps are different from each others.</li></ul><blockquote>maxpool in the image is a type of Pooling layer.</blockquote><p>Here is a snapshot of feature maps from conv1_1(first layer in block 1) in VGG19 with polar bear image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*Ac53pLRjXzbOMpDYuPrKSw.jpeg" /></figure><p>For conv5_1(first layer in block 5)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*LNKL9xOfdHVWJiKMmuBkRQ.jpeg" /></figure><p>Features from an image are aggregated and more clear when it approach to the end of block. This is the how CNN learn features to classify images.</p><p>The feature maps is an important element in Neural Style Transfer.</p><blockquote>Counting Convolutional layer from conv1_1 to conv5_4 and add 3 Fully-connected layers is 19. Therefore it is named VGG19.</blockquote><h3>Neural Style Transfer</h3><p>Here I am using VGG19 for Neural Style Transfer. There are 3 images will be passed through VGG19 model then get output of feature maps from Convolutional layer to calculate style and content loss. Finally update generated image with loss of style and content.</p><blockquote>Generated image here means final produced image with style neither style image nor content image.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*N3RVnzRJKwmWKqxwHjfaew.png" /><figcaption>Neural Style Transfer</figcaption></figure><h4>3 input images</h4><p>These images will be passed into VGG19 model.</p><ul><li><strong>Style image</strong>: the image represent style</li><li><strong>Content image</strong>: the image represent original content</li><li><strong>Generated image</strong>: combination of style and content images</li></ul><blockquote>In the beginning, generated image can either be a white noise image or a copy of content image. And it be updated toward final image which is combination of style and content throughout training process.</blockquote><h4>Content loss</h4><p>Get the feature maps from <em>conv5_2</em>(layer 2 in block 5) for both content image and generated image. And calculate content loss with MSE(Mean Square Error) loss.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*eCGP6vDo0knct8X2_km-Pg.jpeg" /><figcaption>Mean Square Error</figcaption></figure><h4>Style loss</h4><p>Compute correlation across the different feature maps by <strong>gram matrix</strong> which include correlation information.</p><p>First get the feature maps from <em>conv1_1</em>, <em>conv2_1</em>, <em>conv3_1, conv4_1 </em>and <em>conv5_1</em> for both style image and generated image. Second calculate <strong>gram matrix</strong> for each of them then calculate loss with MSE. Finally add up their loss. And that is style loss.</p><h4>Gram Matrix</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FFQiNeuRqZu8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DFQiNeuRqZu8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FFQiNeuRqZu8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/11202b90144fc8ef3afbda3a2e6d397c/href">https://medium.com/media/11202b90144fc8ef3afbda3a2e6d397c/href</a></iframe><p>With gram matrix, we are able to find the correlation between the different feature maps. Remember when calculating gram matrix across the different feature maps, every feature maps need to be flatten. For example in order to calculate gram matrix of a feature maps of shape [32, 224, 224](feature, height, width), it need to be turned into shape of [32, 50176](feature, height*width).</p><h3>Implementation</h3><p>I am using PyTorch for this task. Here is the <a href="https://drive.google.com/file/d/1X1pkP8K9WsEsOknXlW0D1_lSCepIbCbR/view?usp=sharing"><strong><em>code</em></strong></a> written in Colab.</p><p>The implementation is base on this <a href="https://pytorch.org/tutorials/advanced/neural_style_tutorial.html#gradient-descent"><em>article</em></a>.</p><h4>Setup</h4><p>Import necessary libraries, download content and style images and setup device to work with.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e3257f4ff05d36e93d338e80ee2f0b1/href">https://medium.com/media/7e3257f4ff05d36e93d338e80ee2f0b1/href</a></iframe><h4>Load and visualize image</h4><p>For loading and prepare our images for content and style.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/247adae97d1498864ec628e3620d8c24/href">https://medium.com/media/247adae97d1498864ec628e3620d8c24/href</a></iframe><p>Define a function <strong><em>load_image </em></strong>and this function use PyTorch’s <strong><em>read_image</em></strong> utility to read image from file paths then transform image and return a list of transformed images. In addition the function also remove alpha color channel(line 35) from input image since we expect images in RGB and add 1 dimension(line 36) to image, this extra dimension is batch dimension.</p><p>In order to visualize our prepared images.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dfd70b883115ecc049e338d59862ebeb/href">https://medium.com/media/dfd70b883115ecc049e338d59862ebeb/href</a></iframe><p>Define a function <strong><em>show_images </em></strong>to display a set of PyTorch’s images. Line 31 is where this function swap dimensions on image because PyTorch’s <strong><em>read_image</em></strong> utility return image as [channel, height, width], in order to properly display image we need to convert it to [height, width, channel].</p><h4>VGG19 model</h4><p>We need to create our model from VGG19 because some layers in VGG19 are not included for this style transfer task. Beside we need to create style loss and content loss layer and attach them after convolutional layers.</p><p>Load VGG19 model and freeze its layers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af1370e4684af442e6471032ae3d9ab2/href">https://medium.com/media/af1370e4684af442e6471032ae3d9ab2/href</a></iframe><p>Load VGG19 model with default pre-trained weight and <strong><em>excluded</em></strong> <strong><em>fully-connected</em></strong> layers <strong><em>.features.eval()</em></strong> then make sure layers’ gradient is disabled.</p><h4><em>Gram Matrix</em>, <em>Content loss layer</em>, <em>Style loss layer</em> and <em>Normalization layer</em></h4><p><strong>Gram Matrix</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/04543b752f9f20058e8e6ae5e14049cb/href">https://medium.com/media/04543b752f9f20058e8e6ae5e14049cb/href</a></iframe><p>Line 12 we transform input tensor into 2 dimensional vectors where first dimension represent number of feature maps and second dimension is flatten features. At line 13 calculate gram matrix with matrix multiplication. Finally line 14 normalize gram matrix and return the result.</p><p><strong>Content loss</strong></p><p>We need to create a content loss layer to track loss value.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4086c0ce8556bc815dcb603b5f4c843/href">https://medium.com/media/e4086c0ce8556bc815dcb603b5f4c843/href</a></iframe><p>In <strong><em>initialization</em></strong>, we capture target features that are output from convolutional layer after content image been fed to model. And it will be used for calculating the loss.</p><p>In <strong><em>forward</em></strong>, we calculate loss between <strong><em>input</em></strong> and target features by <strong><em>MSE</em></strong> loss function.</p><blockquote>input here is the feature maps from convolutional layer after input image been fed to model.</blockquote><p><strong>Style loss</strong></p><p>Similar to content loss layer.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7fe1a9433befdd117ba4a2bdd675d39/href">https://medium.com/media/d7fe1a9433befdd117ba4a2bdd675d39/href</a></iframe><p>Instead of using features, we calculate gram matrix of features and then find out the loss.</p><p><strong>Normalization</strong></p><p>According to <a href="https://pytorch.org/tutorials/advanced/neural_style_tutorial.html"><strong>PyTorch’s Neural Style Transfer</strong></a>.</p><blockquote>VGG networks are trained on images with each channel normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225]. We will use them to normalize the image before sending it into the network.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbc6bb7cd0378e0236a0d9e3426ef403/href">https://medium.com/media/cbc6bb7cd0378e0236a0d9e3426ef403/href</a></iframe><h4>Create the model</h4><p>Put everything together to create our model.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cfaeec8510762c10e9b6892acf6ab56f/href">https://medium.com/media/cfaeec8510762c10e9b6892acf6ab56f/href</a></iframe><p>Line 30 ~ 49 we extract each layers from VGG19 model and added them to our new model. In addition we track their names. Line 52 ~ 63 we either add content loss layer or style loss layer if the current layer’s name match any specified name in list.</p><blockquote>Be aware of line 39. ReLU layer is recreated with inplace=False to prevent runtime error because VGG19’s ReLU layer has inplace=True</blockquote><h4>Visualize feature maps</h4><p>For visualizing output feature maps of a particular convolutional layer that was mention early in article.</p><p>Prepare model and forward hook.</p><p>A forward hook is a function that can be attached to a particular layer in a model and will be called after the layer’s <strong><em>forward </em></strong>method is called.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2afc7c1c9089704ef1966a2124cfc30/href">https://medium.com/media/e2afc7c1c9089704ef1966a2124cfc30/href</a></iframe><p><strong><em>get_feature_hook </em></strong>function will return a hook function. In function <strong><em>get_feature_maps </em></strong>we register forward hook to any specified layer in a model.</p><p><strong>Visualize Conv1_1</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/25e4284a2b2aad817abae5575007c363/href">https://medium.com/media/25e4284a2b2aad817abae5575007c363/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*99gzkMBwCwOqeCFes5SCfQ.jpeg" /><figcaption>conv1_1 feature maps</figcaption></figure><p><strong>Visualize Conv5_1</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/916e804efc43af9f96980b1963b5840e/href">https://medium.com/media/916e804efc43af9f96980b1963b5840e/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*D_koA5-jO0acZKawzSI1GA.jpeg" /><figcaption>conv5_1 feature maps</figcaption></figure><h4>Training</h4><p>For Neural Style Transfer, we will use optimizer to optimize the input image not our model. Only thing we do in training is to pass input image into model and calculate the loss of content and style then update input image.</p><p>We will train with 2 different optimizer <strong><em>Adam </em></strong>and <a href="https://discuss.pytorch.org/t/pytorch-tutorial-for-neural-transfert-of-artistic-style/336/20?u=alexis-jacq"><strong><em>LBFGS</em></strong></a>. Beside we will save image at interval during training.</p><p><strong>Setting</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/322040196fee21023814616e2b1674ce/href">https://medium.com/media/322040196fee21023814616e2b1674ce/href</a></iframe><p>We declare some variables for training.</p><p><strong>Save image at interval</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2abdc87a009d0d0f3bb87c6ca1b649ce/href">https://medium.com/media/2abdc87a009d0d0f3bb87c6ca1b649ce/href</a></iframe><p>We define a function to save image during training.</p><p><strong>Train with Adam optimizer</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b1e2ad108641ad73d16b40a3652917b/href">https://medium.com/media/3b1e2ad108641ad73d16b40a3652917b/href</a></iframe><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FXU8cUPlP5bpDtYLni2%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZnZtNG1yMnNiNzA4aGg0Y2hiMXAwcDNoZTl3OHNrN3IzZmMzNzI3eSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FXU8cUPlP5bpDtYLni2%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia3.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZ2F1eHo1c3gzdXA0eG1lYjI2bjZxa3liYXYwNnc5aTltczd5azNoNSZlcD12MV9naWZzX2dpZklkJmN0PWc%2FXU8cUPlP5bpDtYLni2%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="289" frameborder="0" scrolling="no"><a href="https://medium.com/media/0a81fab84ac38508578d46d5dd907c0a/href">https://medium.com/media/0a81fab84ac38508578d46d5dd907c0a/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*47fpGwjAzV38TScoqg6XVg.jpeg" /><figcaption>Adam final</figcaption></figure><p><strong>Train with LBFGS optimizer</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bb9e233d91ef60e2eb9f4c5799fa5fa3/href">https://medium.com/media/bb9e233d91ef60e2eb9f4c5799fa5fa3/href</a></iframe><blockquote>Line 87 input image must be contiguous otherwise it will cause runtime error</blockquote><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Ffj5eOXGBOlUlvVxyrD%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZ2l0dDZkYW1zd2Z2ZG11ODY2eGpid3ZzZDhsZG16MGxvdGdkaWltciZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2Ffj5eOXGBOlUlvVxyrD%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExMTVrNDNlN2xmb3d2emprMDVsZm01d296MHY4ZWt2M2xjMnptcjIzbyZlcD12MV9naWZzX2dpZklkJmN0PWc%2Ffj5eOXGBOlUlvVxyrD%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="289" frameborder="0" scrolling="no"><a href="https://medium.com/media/6e42db009f302ff8b9252feec663aea9/href">https://medium.com/media/6e42db009f302ff8b9252feec663aea9/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*YzTX_YrubzApVOKLEuVjjw.jpeg" /><figcaption>LBFGS final</figcaption></figure><h3>Code</h3><p><a href="https://drive.google.com/file/d/1X1pkP8K9WsEsOknXlW0D1_lSCepIbCbR/view?usp=sharing"><em>Source code</em></a> in Colab.</p><h3>Conclusion</h3><p>This is Neural Style Transfer.</p><p>The key point of applying style on an image is <strong><em>Gram Matrix</em></strong>. We can observe content loss go up while style loss go down during training. In addition we can select different convolutional layers in different block for experiment.</p><p>Neural Style Transfer approach is quite different than GAN. It need a paired images whereas GAN don’t need. The weight can be adjusted in Neural Style Transfer to either lean toward more on content or style.</p><p>If you have small set of images and need more controls then Neural Style Transfer can be an option. GAN is more general and used for large set of images.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dab643ec6160" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/neural-style-transfer-vgg19-dab643ec6160">Neural Style Transfer VGG19</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Brain Tumor Detector part 7]]></title>
            <link>https://medium.com/software-dev-explore/brain-tumor-detector-part-7-b4dbb8685110?source=rss-83260f508b12------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4dbb8685110</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[brain-tumor]]></category>
            <category><![CDATA[streamlit]]></category>
            <category><![CDATA[production]]></category>
            <dc:creator><![CDATA[Nelson Punch]]></dc:creator>
            <pubDate>Thu, 09 Nov 2023 02:48:27 GMT</pubDate>
            <atom:updated>2023-12-18T03:08:19.698Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f5py25BWAyXAsAqe" /><figcaption>Photo by <a href="https://unsplash.com/@elijahdhiett?utm_source=medium&amp;utm_medium=referral">Elijah Hiett</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Introduction</h3><p>Finally I can save the model for production use. Here I am going to save the model as a file and then deploy it on a web app for user to interact with.</p><p><a href="https://streamlit.io/"><strong><em>Streamlit</em></strong></a><strong><em> </em></strong>is a tool for creating a machine learning web app quickly.</p><h3>Save the model</h3><p>I saved the model as <strong><em>keras </em></strong>and <strong><em>h5 </em></strong>format. And I will use <strong><em>h5 </em></strong>later for web app. <a href="https://www.tensorflow.org/tutorials/keras/save_and_load#save_the_entire_model"><strong><em>The difference between keras and h5</em></strong></a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03f176022c4099a23b3337e2319f19c8/href">https://medium.com/media/03f176022c4099a23b3337e2319f19c8/href</a></iframe><p>After they are saved then I can download the model I want from Kaggel’s notebook into local machine. In addition download <strong><em>id_to_class.txt</em></strong> file as well.</p><h3>Streamlit app</h3><h4>Environment</h4><p>It is a good idea to create an ioslated environment for my app at local machine. I can use either</p><ul><li><a href="https://docs.conda.io/en/latest/"><strong><em>Conda</em></strong></a></li><li><a href="https://docs.python.org/3/library/venv.html"><strong><em>Python venv</em></strong></a></li></ul><p>for an isolated environment and install necessary libraries. Libraries which will be used to create streamlit app.</p><h4>Note</h4><p>The library for <strong><em>opencv</em></strong> used <a href="https://pypi.org/project/opencv-python-headless/"><strong><em>opencv-python-headless</em></strong></a> instead of <a href="https://pypi.org/project/opencv-python/"><strong><em>opencv-python</em></strong></a> because in this case I had encountered an error by using <a href="https://pypi.org/project/opencv-python/"><strong><em>opencv-python</em></strong></a><strong><em> </em></strong>after web app is deployed.</p><h4>Model wrapper</h4><p>A wrapper to wrap around the model. There are 2 reasons to create this model wapper.</p><ol><li>Load model and talk to the model within this wapper</li><li>Provide interface for my application</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/44e8061701e2b18aeefcaf96f022a1e5/href">https://medium.com/media/44e8061701e2b18aeefcaf96f022a1e5/href</a></iframe><p>The purpose of this wrapper class is to make prediction from an image. Moreover it preprocessing the input image before making prediction. The preprocessing method Line 72 ~ 101 is as same as the code was written in <a href="https://www.kaggle.com/code/nelsonpunch/efficientnetv2-braintumors/notebook"><strong><em>Kaggle’s notebook</em></strong></a>.</p><p>Lines 21 ~ 30 is where actual prediction happend and it return the following</p><ul><li><strong>cls </strong>: class name</li><li><strong>prob </strong>: probability</li><li><strong>alert </strong>: whether there is a tumor or not</li></ul><p>Lines 36 ~ 38 is a support method for prediction. it take an image data in a form of <a href="https://pillow.readthedocs.io/en/stable/reference/Image.html"><strong><em>pillow image</em></strong></a>. And use method line 65 ~ 70 to turn an image into tensor data. Finally it invoke prediction method and return the result.</p><p>The model’s output is in integer where each unique integer correspond to a certain class name. Lines 48 ~ 55 is where it load the file which contain the reference of index to class name.</p><h4>Localization</h4><p>I could not find an existing library to handle localization for <a href="https://streamlit.io/"><strong><em>Streamlit</em></strong></a>. Therefore to support multiple languages for the web app, I need to create a my library to handle localization.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a3ef0c25425bfce77ec88c89cc9c0cad/href">https://medium.com/media/a3ef0c25425bfce77ec88c89cc9c0cad/href</a></iframe><p>This class simply load any encountered localization file in json format within a direcotry. In addition it support changing language on the fly.</p><p><a href="https://github.com/NPMachineLearning/BrainTumorDetectorStreamlit/tree/master/localizations"><strong><em>Here is examples of the file structure and localization file</em></strong></a>.</p><h4>App</h4><p>For the web app. I put everything together.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d5f72bb52df7abb19501adc90cd51843/href">https://medium.com/media/d5f72bb52df7abb19501adc90cd51843/href</a></iframe><h3>Push to Github</h3><p>Put everything under the same folder and then use <a href="https://git-scm.com/"><strong><em>git</em></strong></a> to do version control, track all files and push to Github.</p><p>Create a repository on Github. Follow streamlit <a href="https://docs.streamlit.io/streamlit-community-cloud/deploy-your-app"><strong><em>instuction </em></strong></a>on how to delploy app with Github.</p><p><strong><em>requirement.txt</em></strong> is a necessary file that need to be created in order to deploy app to streamlit cloud server. To automatically generate <strong><em>requirement.txt</em></strong><em>, </em><a href="https://pypi.org/project/pipreqs/">pipreqs</a><em> </em>library can be used for the purpose.</p><p>The github would look similar to <a href="https://github.com/NPMachineLearning/BrainTumorDetectorStreamlit"><strong><em>this</em></strong></a>.</p><h3>Live app</h3><p>Now I have a live app <a href="https://braintumordetectorapp.streamlit.app/"><strong><em>here</em></strong></a>.</p><h3>Conclusion</h3><p>This is end of brain tumor detector journey from an idea to finished working product. Despite of some prediction errors for identifying the type of tumor, the model can generalize well on unseen real life sample.</p><p>It would be greate to have an auxiliary feature that model can predict the tumor location and mark it on image.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4dbb8685110" width="1" height="1" alt=""><hr><p><a href="https://medium.com/software-dev-explore/brain-tumor-detector-part-7-b4dbb8685110">Brain Tumor Detector part 7</a> was originally published in <a href="https://medium.com/software-dev-explore">Software-Dev-Explore</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>