<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Prabhat Kumar Sahu on Medium]]></title>
        <description><![CDATA[Stories by Prabhat Kumar Sahu on Medium]]></description>
        <link>https://medium.com/@caffeinedev?source=rss-e644b56a185d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ux3sKSIW69WR1e4mr-sZIw.jpeg</url>
            <title>Stories by Prabhat Kumar Sahu on Medium</title>
            <link>https://medium.com/@caffeinedev?source=rss-e644b56a185d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:50:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@caffeinedev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Trying out FerretDB]]></title>
            <link>https://caffeinedev.medium.com/trying-out-ferretdb-9d02e1a28881?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d02e1a28881</guid>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Sun, 26 Mar 2023 19:38:51 GMT</pubDate>
            <atom:updated>2023-03-26T19:38:51.401Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently I was searching for something related to DB and stuff and found out about <a href="https://www.ferretdb.io/">FerrertDB</a> on <a href="https://news.ycombinator.com/item?id=29448906">Hackernews</a>. You can check the thread. It seemed pretty interesting so I thought of writing a blog post on this.</p><p>Here are some of my thoughts and opinion about FerretDB.</p><h3>So What’s FerretDB?</h3><p>Quoting from the <a href="https://docs.ferretdb.io/">docs</a></p><p>FerretDB (formerly MangoDB) is an open-source proxy that translates MongoDB wire protocol queries to SQL, with PostgreSQL as the database engine</p><p>Before jumping into what FerretDB offers, Let’s talk a little about MongoDB and Its licensing terms.</p><h3>MongoDB and SSPL</h3><p>The MongoDB Community Server is licensed under the Server Side Public License (SSPL).</p><p>Before October 2018, MongoDB had AGPL licensing. But they changed their open-source roots and the license to a new, more-restrictive license called <a href="https://www.mongodb.com/licensing/server-side-public-license">SSPL</a>.</p><p>Note: SSPL is not an [OSI](https://opensource.org/osd/)-approved license</p><p>You can read the SSPL in full <a href="https://www.mongodb.com/licensing/server-side-public-license">here</a>, but this is the crucial sentence:</p><p>If you make the functionality of the Program or a modified version available to third parties as a service, you must make the Service Source Code available via network download to everyone at no charge, under the terms of this License.</p><p>Let’s break it down with a simple example</p><ul><li>Let’s say a company named SmallCompany uses MongoDB Community Edition Software for its data store. And the SmallCompany is not making the MongoDB functionality available as a service. As long as the company is not providing MongoDB as a service offering, then there is no problem. It’s a valid use of the license.</li><li>Let’s say another company named BigComapny offers MongoDB as a service. Then this is not the valid use of the license. They need to pay for an Enterprise license or open-source their codebases. Meaning</li></ul><p>You might say, “ Hey I am not concerned, I am not providing DB as a service ”</p><p>But I feel like we all should be more concerned about this.</p><ul><li>We used an open-source project and now that’s not open-source anymore. At least the updates and all. And we will have to stop using that because of that.</li><li>They will have a monopoly. Meaning the company behind these products will be the only one to provide cloud services. This means they will not have any competitors, thus it will result in a price increase and lack of innovation.</li><li>A key aspect of Mongo’s success has been that it started as an open-source project, used that openness to build its community (and its product itself), and then pulled the rug out from under everyone to gain an advantage. MongoDB’s decision to no longer offer a true open-source option for their software is exploitative and a slap in the face to every open-source developer.</li></ul><p>We should ask ourselves more questions</p><ul><li>How can a software organization ask others to give back to the community while they are not using open-source anymore?</li><li>How can a company ask others to open-source its source code, even if it does not publish the source code of its cloud services?</li><li>How can a company declare that it’s open-source while using non-open-source licenses like SSPL?</li></ul><p>I am not an expert but probably a better way would have been</p><ul><li>If the company needs to be profitable, it should provide an enterprise license and premium service support, etc.</li><li>Companies can use copyleft licenses to ensure that modifications they make to code will be shared with the community.</li></ul><p>If a company goes for SSPL and then</p><ul><li>lose many independent contributors and employees of big firms that contributed to your open-source software</li><li>Your product could be associated with an unflattering buzz.</li></ul><p>These are my thoughts and opinion. It may be argumentative but this is what I feel.</p><p>You may read more detailed explanations regarding SSPL and all from the link posted at the end.</p><h3>What does FerretDB solve?</h3><p>If you have previous experience using MongoDB, You can use FerretDB as a proxy layer and use the same existing commands, and queries of MongoDB in your application. FerretDB converts MongoDB wire protocol queries to SQL, using Postgres as its database engine.</p><h3>My experience</h3><p>I thought of giving it a try and build a simple REST-API application using FerretDB. It was just simple CRUD operations. You can find my Github repository here</p><p><a href="https://github.com/thecaffeinedev/rest-api-ferretdb">https://github.com/thecaffeinedev/rest-api-ferretdb</a></p><p>FerretDB is still in the early stage and in active development. I will surely try more complex queries later on.</p><p>Do give it a <a href="https://github.com/FerretDB/FerretDB">star</a>. They are doing some awesome work.</p><p>Thanks</p><h3>Good Reads and References</h3><ul><li><a href="https://www.percona.com/blog/why-is-mongodbs-sspl-bad-for-you/">https://www.percona.com/blog/why-is-mongodbs-sspl-bad-for-you/</a></li><li><a href="https://www.percona.com/blog/why-sspl-is-bad-for-you-part-2/">https://www.percona.com/blog/why-sspl-is-bad-for-you-part-2/</a></li><li><a href="https://www.scylladb.com/2018/10/22/the-dark-side-of-mongodbs-new-license/">https://www.scylladb.com/2018/10/22/the-dark-side-of-mongodbs-new-license/</a></li><li><a href="https://news.ycombinator.com/item?id=29448906">https://news.ycombinator.com/item?id=29448906</a></li><li><a href="https://blog.ferretdb.io/open-source-is-in-danger/">https://blog.ferretdb.io/open-source-is-in-danger/</a></li><li><a href="https://blog.ferretdb.io/5-database-alternatives-mongodb-2023/">https://blog.ferretdb.io/5-database-alternatives-mongodb-2023/</a></li><li><a href="https://igorkotua.com/strong-oss-licenses/">https://igorkotua.com/strong-oss-licenses/</a></li><li><a href="https://blog.opensource.org/the-sspl-is-not-an-open-source-license/">https://blog.opensource.org/the-sspl-is-not-an-open-source-license/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d02e1a28881" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Install TensorFlow on M1 Mac]]></title>
            <link>https://caffeinedev.medium.com/how-to-install-tensorflow-on-m1-mac-8e9b91d93706?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e9b91d93706</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[m1]]></category>
            <category><![CDATA[apple-silicon]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Sun, 13 Jun 2021 15:55:51 GMT</pubDate>
            <atom:updated>2021-10-28T14:13:10.109Z</atom:updated>
            <content:encoded><![CDATA[<h3>How To Install TensorFlow on M1 Mac (The Easy Way)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nzaiZv6656n20mLw8Sv7Vw.jpeg" /><figcaption>Tensorflow</figcaption></figure><p>Last year in November 2020 apple releases their first ARM64-based M1 chip. It got a lot of attention from everyone.</p><p>Being a tech enthusiast and a programmer, I was amazed to see the performance of the new apple M1 chip. The benchmarks were really good.</p><p>Recently only some months back, I bought myself an M1 Macbook Pro with 8Gigs of RAM and 512GB of SSD.</p><p>I wanted to set up and test how machine learning frameworks are working in this new chip. Here are the setup instructions for Tensorflow.</p><p>At this time of writing Apple has released its Tensorflow 2.5 for Apple Silicon.</p><p>It’s very easy to set up.</p><p>Before jumping into, I hope <a href="https://brew.sh/">Homebrew</a> is already installed in your system if not you can install it by running the following in your terminal</p><pre>/bin/bash -c “$(curl -fsSL <a href="https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh">https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh</a>)&quot;</pre><p>Here are the things that we are going to do.</p><ul><li>Step 1: Xcode Command Line Tools</li><li>Step2: Install Miniforge</li><li>Step3: Create a virtual environment with python3.8</li><li>Step4: Install Tensorflow 2.5 and its dependencies</li><li>Step5: Install Jupyter Notebook, Pandas</li><li>Step6: Run a Benchmark by training the MNIST dataset</li></ul><h3>Step1: Install Xcode Command Line Tools</h3><p>I have already installed Xcode Command Line Tools on my mac. If it’s not already installed in your system, you can install it by running the following command below in your terminal.</p><pre>xcode-select --install</pre><h3>Step2: Install Miniforge</h3><p>Install miniforge for arm64 (Apple Silicon) from <a href="https://github.com/conda-forge/miniforge">miniforge GitHub</a>.</p><p>Miniforge enables installing python packages natively compiled for Apple Silicon.</p><p>After the installation of miniforge, by default, it gives us one base environment. You can turn off the default base env by running</p><pre>conda config --set auto_activate_base false</pre><h3>Step3: Create a virtual environment</h3><p>Let’s create a virtual environment named <em>mlp</em> with python3.8 installed.</p><pre>conda create --name mlp python=3.8</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*iMuVAT0AYBYQPrTR9iGr1g.png" /></figure><p>And activate it by running</p><pre>conda activate mlp</pre><h3>Step4: Installing Tensorflow-MacOS</h3><p>Install the Tensorflow dependencies:</p><pre>conda install -c apple tensorflow-deps</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*yRFNBCqlkHbnnnmoLzXtnA.png" /></figure><h4>Install base TensorFlow:</h4><pre>pip install tensorflow-macos</pre><h4>Install metal plugin:</h4><pre>pip install tensorflow-metal</pre><h3>Step5: Install Jupyter Notebook &amp; Pandas</h3><pre>conda install -c conda-forge -y pandas jupyter</pre><h3>Step6: Run a Benchmark by training the MNIST dataset</h3><p>Let’s install Tensorflow Datasets</p><pre>pip install tensorflow_datasets</pre><p>Make sure your conda environment is activated.</p><p>Let’s open a Jupyter Notebook and do the benchmark. In your terminal run</p><pre>jupyter notebook</pre><p>It will open a browser window</p><p>Create a new python3 notebook</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*yFB2wJOaQu-ftdLDZuF3Kg.png" /></figure><p>Let’s first import TensorFlow and check</p><pre>import tensorflow as tf</pre><pre>print(&quot;Num GPUs Available: &quot;, len(tf.config.experimental.list_physical_devices(&#39;GPU&#39;)))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*PaQ7TvVlatf2TGjIFQj9Tg.png" /></figure><p>Let’s run a benchmark</p><p>I got the code snippet from</p><p><a href="https://github.com/apple/tensorflow_macos/issues/25">https://github.com/apple/tensorflow_macos/issues/25</a></p><p>Copy and paste this code below in the new notebook</p><pre>%%time</pre><pre>import tensorflow as tf</pre><pre>import tensorflow_datasets as tfds</pre><pre>print(&quot;TensorFlow version:&quot;, tf.__version__)<br>print(&quot;Num GPUs Available: &quot;, len(tf.config.experimental.list_physical_devices(&#39;GPU&#39;)))<br>tf.config.list_physical_devices(&#39;GPU&#39;)</pre><pre>(ds_train, ds_test), ds_info = tfds.load(<br>    &#39;mnist&#39;,<br>    split=[&#39;train&#39;, &#39;test&#39;],<br>    shuffle_files=True,<br>    as_supervised=True,<br>    with_info=True,<br>)</pre><pre>def normalize_img(image, label):<br>  &quot;&quot;&quot;Normalizes images: `uint8` -&gt; `float32`.&quot;&quot;&quot;<br>  return tf.cast(image, tf.float32) / 255., label</pre><pre>batch_size = 128</pre><pre>ds_train = ds_train.map(<br>    normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE)<br>ds_train = ds_train.cache()<br>ds_train = ds_train.shuffle(ds_info.splits[&#39;train&#39;].num_examples)<br>ds_train = ds_train.batch(batch_size)<br>ds_train = ds_train.prefetch(tf.data.experimental.AUTOTUNE)</pre><pre>ds_test = ds_test.map(<br>    normalize_img, num_parallel_calls=tf.data.experimental.AUTOTUNE)<br>ds_test = ds_test.batch(batch_size)<br>ds_test = ds_test.cache()<br>ds_test = ds_test.prefetch(tf.data.experimental.AUTOTUNE)</pre><pre>model = tf.keras.models.Sequential([<br>  tf.keras.layers.Conv2D(32, kernel_size=(3, 3),<br>                 activation=&#39;relu&#39;),<br>  tf.keras.layers.Conv2D(64, kernel_size=(3, 3),<br>                 activation=&#39;relu&#39;),<br>  tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),<br>#   tf.keras.layers.Dropout(0.25),<br>  tf.keras.layers.Flatten(),<br>  tf.keras.layers.Dense(128, activation=&#39;relu&#39;),<br>#   tf.keras.layers.Dropout(0.5),<br>  tf.keras.layers.Dense(10, activation=&#39;softmax&#39;)<br>])<br>model.compile(<br>    loss=&#39;sparse_categorical_crossentropy&#39;,<br>    optimizer=tf.keras.optimizers.Adam(0.001),<br>    metrics=[&#39;accuracy&#39;],<br>)</pre><pre>model.fit(<br>    ds_train,<br>    epochs=12,<br>    validation_data=ds_test,<br>)</pre><p>You will get the output below. The time to run may be different</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dP1vjUvR_ttkSXhPIhQpSw.png" /></figure><p>It took a total of 1min 57s second to run 12 epochs with 128 batch size.</p><p>You can use any other code editor or any other way to do the test benchmark also.</p><p><strong>Conclusion</strong></p><p>As you can see Tensorflow is successfully installed in our system and we are able to run some code also.</p><p>If you have any questions, recommendations, or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:iprabhatdev@gmail.com">mail</a>. Feel free to reach out to me.</p><h3>References:</h3><ul><li><a href="https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html">https://blog.tensorflow.org/2020/11/accelerating-tensorflow-performance-on-mac.html</a></li><li><a href="https://developer.apple.com/metal/tensorflow-plugin/">https://developer.apple.com/metal/tensorflow-plugin/</a></li><li><a href="https://github.com/apple/tensorflow_macos">https://github.com/apple/tensorflow_macos</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e9b91d93706" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building and Installing OpenCV on M1 Macbook]]></title>
            <link>https://caffeinedev.medium.com/building-and-installing-opencv-on-m1-macbook-c4654b10c188?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4654b10c188</guid>
            <category><![CDATA[m1]]></category>
            <category><![CDATA[m1-chip]]></category>
            <category><![CDATA[opencv]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Thu, 27 May 2021 20:23:28 GMT</pubDate>
            <atom:updated>2021-06-18T14:28:11.421Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*NCjd8eLzfiZwqK5JnfaR9A.png" /></figure><p>In this blog post, we are going to build and install OpenCV on anM1 Mac.</p><p>Last year in November 2020 apple releases their first ARM64-based M1 chip. It got a lot of attention from everyone.</p><p>Being a tech enthusiast and a programmer, I was amazed to see the performance of the new apple M1 chip. The benchmarks were really good.</p><p>Recently only some months back, I bought myself an M1 Macbook Pro with 8Gigs of RAM and 512GB of SSD.</p><p>Let’s get started.</p><p>Here are the things that we are going to do.</p><ul><li>Step 1: Homebrew and Xcode Command Line Tools Installation</li><li>Step2: Install Miniforge</li><li>Step3: Create a virtual environment with python3.8</li><li>Step4: Install necessary dependencies</li><li>Step5: Compile and Build OpenCV</li><li>Step6: Test OpenCV</li></ul><h3>Step1: Homebrew and XCode CLI Installation</h3><p>If <a href="https://brew.sh/">Homebrew</a> is not already installed in your system you can install it by running the following in your terminal.</p><pre>/bin/bash -c &quot;$(curl -fsSL <a href="https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh">https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh</a>)&quot;</pre><p>After it gets installed, install wget and CMake.</p><pre>brew install wget cmake</pre><p>I have already installed Xcode Command Line Tools on my mac. If it’s not already installed in your system, you can install it by running the following command below in your terminal.</p><pre>xcode-select --install</pre><h3>Step2: Install Miniforge</h3><p>Install miniforge for arm64 (Apple Silicon) from <a href="https://github.com/conda-forge/miniforge">miniforge GitHub</a>.</p><p>Miniforge enables installing python packages natively compiled for Apple Silicon.</p><p>After the installation of miniforge, by default, it gives us one base environment. You can turn off the default base env by running</p><pre>conda config --set auto_activate_base false</pre><h3>Step3: Create a virtual environment</h3><p>Let’s create a virtual environment named cv2 with python3.8 installed.</p><pre>conda create --name testcv python=3.8</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NIKNl9-7FvfL3go-NPR8uQ.png" /></figure><p>And activate it by running</p><pre>conda activate testcv</pre><h3>Step4: Installing Some Dependencies</h3><p>As we are going to install OpenCV natively we have to install the compatible version of Numpy which Apple provides for M1.</p><p>Run the following commands below on a terminal to install Numpy and some other dependencies.</p><pre>conda install -c apple tensorflow-deps</pre><h3>Step5: Compile and Build OpenCV</h3><p>We have to download OpenCV and OpenCV Contrib first.</p><p>I have created one folder named test_opencv where I will be downloading files and build there.</p><pre>wget -O opencv.zip <a href="https://github.com/opencv/opencv/archive/4.5.0.zip">https://github.com/opencv/opencv/archive/4.5.0.zip</a></pre><pre>wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.5.0.zip</pre><pre>unzip opencv.zip &amp;&amp; unzip opencv_contrib.zip</pre><pre><em>cd</em> opencv-4.5.0</pre><pre>mkdir build <strong>&amp;&amp;</strong> <em>cd</em> build</pre><p>Let’s Set up the OpenCV build with CMake:</p><p>Make sure to put your own PATH on OPENCV_EXTRA_MODULES</p><p>Also, make sure to activate your conda environment before running make.</p><pre>conda activate testcv</pre><p>You have to put your Python3 executable path also. You can get it by running</p><pre>echo $CONDA_PREFIX</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IZc7yOIpjLtfaj4KlRU1DA.png" /></figure><p>For eg: My path is</p><p><em>/opt/homebrew/Caskroom/miniforge/base/envs/testcv/</em></p><p>So I have put</p><p><em>/opt/homebrew/Caskroom/miniforge/base/envs/testcv/bin/python3</em></p><p>You have to change and put your own path there.</p><p>Now you can run</p><pre><br>cmake \                                                                -DCMAKE_SYSTEM_PROCESSOR=arm64 \</pre><pre>-DCMAKE_OSX_ARCHITECTURES=arm64 \</pre><pre>-DWITH_OPENJPEG=OFF \</pre><pre>-DWITH_IPP=OFF \</pre><pre>-D CMAKE_BUILD_TYPE=RELEASE \</pre><pre>-D CMAKE_INSTALL_PREFIX=/usr/local \</pre><pre>-D OPENCV_EXTRA_MODULES_PATH=/Users/caffeinedev/Prabhat/Workspace/test_opencv/opencv_contrib-4.5.0/modules \</pre><pre>-D PYTHON3_EXECUTABLE=/opt/homebrew/Caskroom/miniforge/base/envs/testcv/bin/python3 \</pre><pre>-D BUILD_opencv_python2=OFF \</pre><pre>-D BUILD_opencv_python3=ON \</pre><pre>-D INSTALL_PYTHON_EXAMPLES=ON \</pre><pre>-D INSTALL_C_EXAMPLES=OFF \</pre><pre>-D OPENCV_ENABLE_NONFREE=ON \</pre><pre>-D BUILD_EXAMPLES=ON ..</pre><p>Snapshots</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ERmg-sL7tL5SGQfAAcwMg.png" /></figure><p>You will see the screen below after Cmake. The compilation doesn’t take that much time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QgSSYmqBrJ3MP4S7wYjvQA.png" /></figure><p>Now we have the run the <em>make</em> command.</p><p>Here j8 stands for 8 cores. You can do either make or you can adjust the <em>j </em>option<em>. </em>So let’s run</p><pre>make -j8</pre><p>You will get an output like the below. It took around 10 minutes for me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GLl6bTvsk8yjqx-yHORpWQ.png" /></figure><p>Then the last step is to run</p><pre>sudo make install</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqZVMgbuGbe3AKqBCTJ08w.png" /></figure><p>You will get the above after sudo make install.</p><h3>Sym-link OpenCV 4 on macOS to virtual environment site-packages</h3><p>We need to first locate the .so file, which has been generated during the compilation of OpenCV.</p><pre>mdfind cv2.cpython</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*98ZgMVdER2yyTYpADnRLXg.png" /></figure><p>Now, we need to execute the following to sym-link one of the .so files in our current Python virtual environment.</p><p>You can see this from the screenshot. There’s one so file that has been generated inside the build folder. The path to the .so file in my system is</p><p>Also, Please double-check the paths before executing the commands.</p><pre>/Users/caffeinedev/Prabhat/Workspace/test_opencv/opencv-4.5.0/build/lib/python3/cv2.cpython-38-darwin.so</pre><p>Open up another terminal and go to your python environment site-packages directory. In my case it’s</p><pre>cd /opt/homebrew/Caskroom/miniforge/base/envs/testcv/lib/python3.8/site-packages</pre><p>And execute the sym-link</p><pre>ln -s /Users/caffeinedev/Prabhat/Workspace/test_opencv/opencv-4.5.0/build/lib/python3/cv2.cpython-38-darwin.so</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aMaG42tkpAiEQ1KTpWS8_A.png" /></figure><p>This I ran on my second terminal.</p><p>And that’s it. Now we are good to go.</p><p>Open a separate terminal if you want and activate your virtual environment</p><pre>conda activate testcv</pre><p>And just run the below</p><pre>python3</pre><p>And</p><pre>import cv2<br>cv2.__version__</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m-6LenCf8cwKBHyQm0gU-A.png" /></figure><h3>Step6: Test OpenCV by running a program</h3><p>Create a new file named main.py and paste the code below. Put a dummy photo for test purposes in the third line. The code just converts the image to a black-white image.</p><pre>import cv2</pre><pre>image = cv2.imread(&#39;grogu.jpeg&#39;)<br>gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)</pre><pre>cv2.imshow(&#39;Original image&#39;,image)<br>cv2.imshow(&#39;Gray image&#39;, gray)</pre><pre>cv2.waitKey(0)<br>cv2.destroyAllWindows()</pre><p>And run it python3 main.py. Here’s the output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TNiqztLHSSofkScKHTjFkw.png" /></figure><h4>Additional Step</h4><p>If you want to uninstall OpenCV, you can go inside the build folder and run</p><pre>sudo make uninstall</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xH8rWQEkKNgIzvbJtqzLZQ.png" /></figure><p>It will remove OpenCV 4.5.0 from your system. Also, you can delete your conda environment. First, deactivate if it’s activated</p><pre>conda deactivate</pre><pre>conda remove --name testcv --all</pre><p>It will remove completely.</p><p><strong>Conclusion</strong></p><p>We were able to build and install OpenCV 4.5.0 successfully in our M1 Mac and we are able to run some code also.</p><p>If you have any questions, recommendations, or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:iprabhatdev@gmail.com">mail</a>. Feel free to reach out to me.</p><p>Thanks</p><h3>References:</h3><ul><li><a href="https://sayak.dev/install-opencv-m1/">https://sayak.dev/install-opencv-m1/</a></li><li><a href="https://caffeinedev.medium.com/setting-up-tensorflow-on-m1-mac-36fe017ce284">https://caffeinedev.medium.com/setting-up-tensorflow-on-m1-mac-36fe017ce284</a></li><li><a href="https://moeenv.blog/?p=209&amp;lang=en">https://moeenv.blog/?p=209&amp;lang=en</a></li><li><a href="https://github.com/apple/tensorflow_macos">https://github.com/apple/tensorflow_macos</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4654b10c188" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training a Custom Object Detection Model With Yolo-V5]]></title>
            <link>https://medium.com/analytics-vidhya/training-a-custom-object-detection-model-with-yolo-v5-aa9974c07088?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa9974c07088</guid>
            <category><![CDATA[yolov5]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[yolo]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Fri, 01 Jan 2021 19:32:32 GMT</pubDate>
            <atom:updated>2021-01-03T16:53:23.486Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*cqf_w3I8rmEfsog9_M4zWA.png" /></figure><p>Object Detection is a task in computer vision that focuses on detecting objects in images/videos.</p><p>There are various object detection algorithms out there like YOLO (You Only Look Once,) Single Shot Detector (SSD), Faster R-CNN, Histogram of Oriented Gradients (HOG), etc.</p><p>In this article, we are going to use Yolo-V5 to train our custom object detection model. YOLO is one of the most famous object detection models.</p><p>Version 1.0 of YOLO-V5 got released on 27 May 2020 by <a href="https://www.linkedin.com/in/glenn-jocher">Glenn Jocher</a> who is the founder &amp; CEO of <a href="https://github.com/ultralytics">Utralytics</a>. It’s written in PyTorch and it’s available in <a href="https://github.com/ultralytics/yolov5">Github</a>.</p><h4><strong>Prerequisites</strong></h4><p>It’s good to have a basic knowledge of deep learning computer vision. And how to work in a Google Colab environment.</p><h4>Steps Covered in this Tutorial</h4><p>To train our own custom object detector these are the steps to follow</p><ul><li>Preparing the dataset</li><li>Environment Setup: Install YOLOv5 dependencies</li><li>Setup the data and the directories</li><li>Setup the YAML files for training</li><li>Training the model</li><li>Evaluate the model</li><li>Visualize the training data</li><li>Running inference on test images</li><li>Export the weight files for later use</li></ul><h3>Preparing the dataset</h3><p>We are going to use the <a href="https://public.roboflow.com/object-detection/aquarium"><strong>Aquarium Dataset</strong></a> which is available in <a href="https://public.roboflow.com/object-detection">Roboflow Public Object Detection datasets</a>. You can check out more datasets that are there.</p><p>The aquarium dataset consists of 638 images. The images were already labeled by the Roboflow team. It has 7 classes such as fish, jellyfish, penguins, sharks, puffins, stingrays, and starfish, and most images contain multiple bounding boxes.</p><p>To download the dataset you need to create a roboflow account first. It’s very simple and easy.</p><figure><img alt="Source: https://public.roboflow.com/object-detection/aquarium" src="https://cdn-images-1.medium.com/max/839/1*SCY870BMPNewUZ3MGOMihg.png" /><figcaption>Source: <a href="https://public.roboflow.com/object-detection/aquarium">https://public.roboflow.com/object-detection/aquarium</a></figcaption></figure><p>If you want to use your own images you can use annotation tools such as <a href="https://github.com/tzutalin/labelImg">LabelImg</a>, <a href="https://github.com/openvinotoolkit/cvat">CVAT</a> or you can try any large-scale solutions like Scale or AWS Ground Truth.</p><p>When you are annotating yourself make sure to follow the best practices. Check this <a href="https://nanonets.github.io/tutorials-page/docs/annotate">link</a> out for more details.</p><p>After the dataset is prepared then we are all set to set up the environment and train the dataset.</p><h3>Environment Setup</h3><p>Here’s the link to my Notebook: <a href="https://colab.research.google.com/drive/16QCaYzTuHCOF9CQLQYmGNxmtY1xKAIdn?usp=sharing">Google Colab</a></p><p>You need a google account to use Google Colab. You can either use my notebook to train or you can create your own notebook and follow along.</p><p>In Google Colab, you will receive a free GPU for 12 hours. If you use a new notebook in Colab change the runtime session to GPU.</p><p>If you are planning to use my notebook then make sure to File → save a copy in your drive. Then you will be able to edit the code.</p><p><strong>Installation of the dependencies</strong></p><pre>!git clone https://github.com/ultralytics/yolov5  # clone repo!pip install -U -r yolov5/requirements.txt  # install dependencies</pre><p>Somehow the PyTorch version didn’t get compatible with the GPU so I installed another version of PyTorch by</p><pre>#installing for google colab GPU use</pre><pre>!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f <a href="https://download.pytorch.org/whl/torch_stable.html">https://download.pytorch.org/whl/torch_stable.html</a><br></pre><p>.</p><pre>%cd /content/yolov5<br>!ls</pre><p>We can import and take a look at our GPU Specification provided by Google Colab.</p><pre>import torch</pre><pre>from IPython.display import Image  # for displaying images</pre><pre>from utils.google_utils import gdrive_download  # for downloading models/datasets</pre><pre>print(&#39;Using torch %s %s&#39; % (torch.__version__, torch.cuda.get_device_properties(0) if torch.cuda.is_available() else &#39;CPU&#39;))<br></pre><p>Here’s what I got</p><pre>Using torch 1.6.0+cu101 _CudaDeviceProperties(name=&#39;Tesla T4&#39;, major=7, minor=5, total_memory=15079MB, multi_processor_count=40)</pre><p>Google Colab comes preinstalled with Cuda and Torch and some other dependencies. If you are planning to train locally then you will have to setup Cuda and the dependencies on your own. I will surely make a tutorial about it later on.</p><h3>Setup the data and the directories</h3><p>After the environment set up is done. We can import the dataset into colab. As I am using the Roboflow dataset I will be downloading, if you plan to use your own you can import it using Google Drive.</p><pre># You need to sign up in roboflow to get the key and then you can use the dataset</pre><pre>!curl -L “https://public.roboflow.com/ds/PUT YOUR OWN KEY HERE” &gt; roboflow.zip; unzip roboflow.zip; rm roboflow.zip</pre><p>This will download the data, unzip and save it inside the yolov5 directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mqGSVF2Hr9eSJH-fh6u4dw.png" /></figure><p><strong>Project Folder structure</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/327/1*o61Ydwo-W0UTviM_p9uBWw.png" /></figure><h3>Setup the YAML files for training</h3><p>To train a YOLO-V5 model, we need to have two YAML files.</p><p>The first YAML to specify:</p><ul><li>where our training and validation data is</li><li>the number of classes that we want to detect</li><li>and the names corresponding to those classes</li></ul><p>This YAML of ours looks like this:</p><pre>train: ./train/images <br>val: ./valid/images  </pre><pre>nc: 7 <br>names: [&#39;fish&#39;, &#39;jellyfish&#39;, &#39;penguin&#39;, &#39;puffin&#39;, &#39;shark&#39;, &#39;starfish&#39;, &#39;stingray&#39;]</pre><p>The second YAML is to specify the whole model configuration. You can change the network architecture in this step if you want but we will go with the default one.</p><p>The YAML which we term custom_yolov5s.yaml:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c70542eb805ca99accf47a80fd4de45d/href">https://medium.com/media/c70542eb805ca99accf47a80fd4de45d/href</a></iframe><p>We can put the YAML file anywhere we want because we can reference the file path later on. But it’s a good idea to put it inside the YoloV5 directory.</p><h3>Training the model</h3><p>After the configuration is done we can begin our training.</p><p>There are multiple hyperparameters that we can specify which are:</p><ul><li><strong>img:</strong> define input image size</li><li><strong>batch:</strong> determine batch size</li><li><strong>epochs:</strong> define the number of training epochs.</li><li><strong>data:</strong> set the path to our YAML file</li><li><strong>cfg:</strong> specify our model configuration</li><li><strong>weights:</strong> specify a custom path to weights</li><li><strong>name:</strong> result names</li><li><strong>nosave:</strong> only save the final checkpoint</li><li><strong>cache:</strong> cache images for faster training</li></ul><p>We need to specify the path of both YAML files which we created above.</p><pre><br>%cd /content/yolov5/</pre><pre>!python train.py --img 416 --batch 80 --epochs 100 --data &#39;./data.yaml&#39; --cfg ./models/custom_yolov5s.yaml --weights &#39;&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dOzYusJMMAy766JT1QPGnQ.gif" /><figcaption>Training</figcaption></figure><p>With 100 epochs the training got completed within 35 minutes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZtvE6f_rqG-0l2fc866wxg.png" /></figure><h3>Evaluate the model</h3><p>Training losses and performance metrics are saved to Tensorboard and also to a logfile defined above with the <strong>— name</strong> flag when we train. In our case, we named this yolov5s_results. (If given no name, it defaults to results.txt.) The results file is plotted as a png after training completes.</p><p>Partially completed results.txt files can be plotted with from utils.utils import plot_results; plot_results().</p><pre># Start tensorboard<br># Launch after you have started training to all the graphs needed for inspection<br># logs save in the folder &quot;runs&quot;</pre><pre>%load_ext tensorboard<br>%tensorboard --logdir /content/yolov5/runs</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H7YQvi6jiEDOBdKVj1o75Q.png" /></figure><h3>Visualize the training data</h3><p>After training starts, view train*.jpg images to see training images, labels, and augmentation effects. We can visualize both Ground Truth Training data, as well as Ground Truth, Augmented data.</p><pre># first, display our ground truth data<br># The ground truth [Train data] is available in jpg file at location /content/yolov5/runs/train/exp2/test_batch0_labels.jpg<br></pre><pre>print(&quot;GROUND TRUTH TRAINING DATA:&quot;)<br>Image(filename=&#39;/content/yolov5/runs/train/exp2/test_batch0_labels.jpg&#39;, width=900)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*JvWJw09FBkZ27i3PY2_a1g.png" /></figure><pre># print out an augmented training example<br># Below is the augmented training data.<br># NOTE: The dataset already contains the augmented data with annotations, so that you dont have to do it.</pre><pre>print(&quot;GROUND TRUTH AUGMENTED TRAINING DATA:&quot;)<br>Image(filename=&#39;/content/yolov5/runs/train/exp2/train_batch0.jpg&#39;, width=900)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*JzuN94gto3d0Si1_lDplAA.png" /></figure><h3>Running inference</h3><p>Using the final trained weight which got saved after training we can run our inference</p><p><strong>Weights folder</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/428/1*NjH-Z26sbfkLA4H1-63qJA.png" /></figure><p>To run the model inference we can use the following command.</p><pre># use the best weights!<br># Final weights will be by-default stored at /content/yolov5/runs/train/exp2/weights/best.pt</pre><pre>%cd /content/yolov5/</pre><pre>!python detect.py --weights<br>/content/yolov5/runs/train/exp2/weights/best.pt --img 416 --conf 0.4 --source ./test/images</pre><ul><li>— source: input images directory or single image path or video path</li><li>— weights: trained model path</li><li>— conf: confidence threshold</li></ul><p>This will process the input and store the output in our inference directory.</p><p>Here are some output images:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aUHDgsewTLtl4pP98rskNA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*jy6bq-oODi0dXuRKJQectg.png" /></figure><h3>Export the weights for later use</h3><p>Now that we have successfully trained our custom model. We can download the weight files and save them in our local directory or in Google Drive.</p><p>To do so we import a Google Drive module and send them out</p><pre>from google.colab import drive<br>drive.mount(&#39;/content/gdrive&#39;)</pre><pre>%cp /content/yolov5/runs/train/exp2/weights/best.pt /content/gdrive/My\ Drive</pre><h3>Conclusion</h3><p>I hope you were able to follow along and was able to train successfully.</p><p>I have uploaded the notebook, config files, and weight to my Github repository. You can check it out <a href="https://github.com/TheCaffeineDev/YoloV5-Custom-Object-Detection">here</a>.</p><p>If you have any questions, recommendations, or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:thedlmonk@gmail.com">mail</a>. Feel free to reach out to me.</p><h3>References:</h3><ul><li>Official Repo: <a href="https://github.com/ultralytics/yolov5">https://github.com/ultralytics/yolov5</a></li><li><a href="https://public.roboflow.com/object-detection/aquarium">Roboflow dataset</a></li><li><a href="https://colab.research.google.com/drive/16QCaYzTuHCOF9CQLQYmGNxmtY1xKAIdn?usp=sharing">My Colab Notebook</a></li><li><a href="https://blog.roboflow.com/how-to-train-yolov5-on-a-custom-dataset/">Roboflow Blog</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa9974c07088" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/training-a-custom-object-detection-model-with-yolo-v5-aa9974c07088">Training a Custom Object Detection Model With Yolo-V5</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Setup and Configure Your Own GitLab Server on Ubuntu 20.04]]></title>
            <link>https://caffeinedev.medium.com/how-to-setup-and-configure-your-own-gitlab-server-on-ubuntu-20-04-73214cf63882?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/73214cf63882</guid>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[ubuntu-20-04]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Fri, 21 Aug 2020 19:27:53 GMT</pubDate>
            <atom:updated>2020-08-21T19:27:53.321Z</atom:updated>
            <content:encoded><![CDATA[<p>I have created a video tutorial. You can watch that for your reference.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYy2FRJwRT9Y%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYy2FRJwRT9Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYy2FRJwRT9Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/124c04fe10b64828b3a86feb1a256ae7/href">https://medium.com/media/124c04fe10b64828b3a86feb1a256ae7/href</a></iframe><p>GitLab CE(Community Edition) is a free and opensource web-based git manager tool &amp; it’s written in Ruby.</p><p>GitLab CE is generally used for Software development teams, it has a lot of features like version control, code review, monitoring, CI &amp; CD, issue management, wiki, etc.</p><p>GitLab allows you to host your Git repository that can be accessed from either your local LAN / WAN / local server or (if you have an available public IP address) from outside your company. The community edition is open-sourced and freely available to install and setup.</p><p>There are three different editions of GitLab available, Community Edition (CE), Enterprise Edition (EE), and a GitLab-hosted version.</p><p>In this article, we will go step by step to install the latest version of GitLab CE on Ubuntu 20.04 / 18.04 Server.</p><p>Below should be your laptop or desktop minimum configuration:</p><ol><li>RAM: Gitlab recommends 4 GB for Gitlab Setup</li><li>2 CPU cores.</li><li>(optional) Domain or subdomain pointing to the server IP address.</li></ol><h3>Installation</h3><p>Login to your Ubuntu 20.04 / 18.04 system and run the following commands in the terminal to install GitLab dependencies.</p><p>The first thing you need to run is the update or upgrade.</p><pre>sudo apt update</pre><p>Then install the necessary dependencies</p><pre>sudo apt-get install -y curl openssh-server ca-certificates</pre><p>If you want GitLab to send notification emails you can either install Postfix from the command below or if you want to use another transactional mail service such as SendGrid, MailChimp, etc you can configure using GitLab SMTP settings after installation.</p><pre>sudo apt-get install -y postfix</pre><p>During the installation on Postfix, you will be asked how to configure Postfix. if you choose Internet site, then enter either the domain or the IP address of the server. Or you can always select Local Only Postfix configuration.</p><p>Check this <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-postfix-as-a-send-only-smtp-server-on-ubuntu-18-04">digital ocean</a> blog for more information regarding how to setup Postfix.</p><h3>Installing GitLab CE</h3><pre>curl -sS <a href="https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh">https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh</a> | sudo bash</pre><p>Once it is done and the repository is enabled you can install GitLab CE package by using this command below</p><pre>sudo apt-get install gitlab-ce</pre><p>If you want to set up using your server address, do the below</p><pre>sudo EXTERNAL_URL=&quot;http://gitlabce.example.com&quot; apt-get install gitlab-ce</pre><p>Once you have the package installed, you can run the provided configuration utility. It provides an automatic configuration. You can modify things later if you need to.</p><pre>sudo gitlab-ctl reconfigure</pre><pre>gitlab-ctl start</pre><p>When the configuration is done, your server will be ready to use.</p><h3>Access GitLab through the Web Interface</h3><p>Open your web browser and type the URL that you mentioned while installing or check your local IP address and enter in the browser you will get GitLab web interface.</p><p>Eg:</p><pre>https://your_gitlab_domain_or_server_IP</pre><p>Once you are done, you will get a screen to register and log in. Then you can easily follow up and configure.</p><p>If you want to uninstall GitLab CE completely run this command below</p><pre>sudo apt-get remove gitlab-ce<br><br>sudo rm -rf /var/opt/gitlab</pre><p>Kill all live process</p><pre>sudo pkill -f gitlab</pre><p>Remove paths</p><pre>sudo rm -rf /opt/gitlab<br><br>sudo rm -rf /etc/gitlab<br><br>sudo rm -rf /var/opt/gitlab</pre><p>Then restart your machine.</p><p>Thanks</p><p>If you have any questions, recommendations, or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:prabhat1015@gmail.com">mail</a>. Feel free to reach out to me.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73214cf63882" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to run VSCode in the browser?]]></title>
            <link>https://caffeinedev.medium.com/how-to-run-vscode-in-the-browser-adb37d586205?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/adb37d586205</guid>
            <category><![CDATA[vscode]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[code]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Mon, 27 Jan 2020 14:45:31 GMT</pubDate>
            <atom:updated>2020-01-27T14:45:31.191Z</atom:updated>
            <content:encoded><![CDATA[<p>I have been into this programming field since my college days and I have tried a lot of editors over time. As a developer, I believe a code editor is one of the most important tool that a developer can have.</p><p>Before I was happy with Sublime Text and Atom for most of the part because it could do everything I needed in my tech stack.</p><p>I switched to VS Code for the following reasons:</p><ol><li><strong>Git integration</strong>: I actually prefer to use Terminal for anything to do with Git but the VS Code git system is really great.</li><li><strong>Integrated terminals</strong>: You get a complete terminal emulator, out of the box.</li><li><strong>Speed &amp; Extensions: </strong>VS Code is not as fast as Sublime Text but it feels faster than Atom. It has thousands of extensions available that cover pretty much every use case I can think of.</li><li><strong>IntelliSense:</strong> Intellisense<strong> </strong>is incredible.VS Code has more intelligent language support, autocompletion, and suggestion. It’s super helpful, sometimes I don’t have to look up various functions and methods.</li><li><strong>Completely Free: </strong>It is open source and updates are very much frequent.</li></ol><p>And there’s more.</p><p>Anyway, this blog post is about how to run VS Code in your browser. I am using Ubuntu 18.04 btw.</p><p>First go to this URL</p><p><a href="https://github.com/cdr/code-server/releases/tag/2.1698">https://github.com/cdr/code-server/releases/tag/2.1698</a> and download the binary file for Linux.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*3d4QSZSQtmP9Q0JS2aAP5Q.png" /></figure><p>Then unzip it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*WfaOMKa2FvtUBY5CQm1j9A.png" /></figure><p>Then go inside the unzipped folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*ehQhkcBsbCOmv_mH86e_Zw.png" /></figure><p>Inside that folder, the code-server is the binary file and you can run it by</p><pre>./code-server</pre><p>As you can see it’s running on my localhost.</p><p>If you want your own password, you can add your own password by</p><pre>PASSWORD=prabhat ./code-server --host 0.0.0.0 --port 8080</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*f5MZua0HuUodU35Epy2ULA.png" /></figure><p>You can change your port if you want to.</p><p>Then you have to go to your local address. In my case it’s</p><pre><a href="http://127.0.0.1:8080/login">http://127.0.0.1:8080/</a></pre><p>It will ask you to enter your password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*DKT4V7o9twkespztuAN19A.png" /></figure><p>After entering your password, you will see your code server up and running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gF82uejatgsAJ990LQDmBQ.png" /></figure><p>So that’s it.</p><p>I think running VS code on my browsers saves me time and makes me more productive.</p><p>And I think a code editor is simply a tool. It doesn’t matter whether you choose Sublime Text or VS Code or any other. What matters is it works for you.</p><p>Thanks for reading. Did this article help you out? If it did, I hope you consider sharing it. You might help someone else out. Thank you so much!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adb37d586205" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Customize your terminal (Oh My Zsh) on Ubuntu 18.04 LTS]]></title>
            <link>https://caffeinedev.medium.com/customize-your-terminal-oh-my-zsh-on-ubuntu-18-04-lts-a9b11b63f2?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/a9b11b63f2</guid>
            <category><![CDATA[oh-my-zsh]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[terminal]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Sat, 04 Jan 2020 08:32:08 GMT</pubDate>
            <atom:updated>2020-08-20T07:42:40.441Z</atom:updated>
            <content:encoded><![CDATA[<h3>Customize your terminal (Oh My Zsh) on Ubuntu 20.04 LTS</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*_z3cLTQ7p2st-PaYda7OXA.png" /></figure><p>In this blog post, I will cover how to install Oh My Zsh and How to customize your terminal and make it fancy.</p><h4>Update the packages</h4><pre>sudo apt-get update<br>sudo apt upgrade</pre><h4>Install prerequisite packages (ZSH, powerline &amp; powerline fonts)</h4><pre>sudo apt install zsh<br>sudo apt-get install powerline fonts-powerline</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*S8VYA_Ks7YqLYK6pFPK17Q.png" /></figure><h4>Clone the Oh My Zsh Repo</h4><pre>git clone <a href="https://github.com/robbyrussell/oh-my-zsh.git">https://github.com/robbyrussell/oh-my-zsh.git</a> ~/.oh-my-zsh</pre><h4>Create a New ZSH configuration file</h4><pre>cp ~/.oh-my-zsh/templates/zshrc.zsh-template ~/.zshrc</pre><h4>Install PowerLevel9k!</h4><pre>git clone <a href="https://github.com/bhilburn/powerlevel9k.git">https://github.com/bhilburn/powerlevel9k.git</a> ~/.oh-my-zsh/custom/themes/powerlevel9k</pre><h4>Set up a theme for your Terminal — Open .zshrc File using gedit editor</h4><pre>gedit .zshrc</pre><p>And change and put these lines :</p><pre>ZSH_THEME=&quot;powerlevel9k/powerlevel9k&quot;</pre><pre>POWERLEVEL9K_DISABLE_RPROMPT=true<br>POWERLEVEL9K_PROMPT_ON_NEWLINE=true<br>POWERLEVEL9K_MULTILINE_LAST_PROMPT_PREFIX=&quot;▶&quot;<br>POWERLEVEL9K_MULTILINE_FIRST_PROMPT_PREFIX=&quot;&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/1*_hi0hSXi8tXqWirDCMU5nQ.png" /></figure><h4>Change your Default Shell</h4><pre>chsh -s /bin/zsh</pre><p>for this command to be effective you need to restart your system.</p><h3>Oh My Zsh Terminal</h3><p>After restarting your terminal will look like this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*gW98D-UxTS7m1IAC-pMtxw.png" /></figure><h4>If you want Syntax Highlighting</h4><p>Clone the ZSH Syntax Highlighting</p><pre>git clone https://github.com/zsh-users/zsh-syntax-highlighting.git &quot;$HOME/.zsh-syntax-highlighting&quot; --depth 1</pre><p>Add syntax-highlighting in .zshrc Configuration</p><pre>echo &quot;source $HOME/.zsh-syntax-highlighting/zsh-syntax-highlighting.zsh&quot; &gt;&gt; &quot;$HOME/.zshrc&quot;</pre><p>And If anything goes wrong, you can revert back to your default shell by</p><pre>chsh -s /bin/bash</pre><p>If you are a data science enthusiast and using anaconda then you need to add these lines below in zshrc file.</p><p>Open a terminal and</p><pre>gedit .zshrc</pre><p>add these. Make sure to change your username after home.</p><pre># &gt;&gt;&gt; conda initialize &gt;&gt;&gt;<br># !! Contents within this block are managed by ‘conda init’ !!<br>__conda_setup=”$(‘/home/yourusername/anaconda3/bin/conda’ ‘shell.bash’ ‘hook’ 2&gt; /dev/null)”<br>if [ $? -eq 0 ]; then<br> eval “$__conda_setup”<br>else<br> if [ -f “/home/yourusername/anaconda3/etc/profile.d/conda.sh” ]; then<br> . “/home/yourusername/anaconda3/etc/profile.d/conda.sh”<br> else<br> export PATH=”/home/yourusername/anaconda3/bin:$PATH”<br> fi<br>fi<br>unset __conda_setup<br># &lt;&lt;&lt; conda initialize &lt;&lt;&lt;</pre><p>Mine looks like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*CtxgScchhgnV_bVhXiz0eg.png" /></figure><p>Then do</p><pre>source ./zshrc</pre><p>There is a lot of customization available. You can check Oh my zsh and Powerlevel9k repository.</p><p><strong>Acknowledgment</strong></p><ol><li>Oh My Zsh — <a href="https://github.com/ohmyzsh/ohmyzsh">https://github.com/ohmyzsh/ohmyzsh</a></li><li><a href="https://github.com/Powerlevel9k/powerlevel9k">https://github.com/Powerlevel9k/powerlevel9k</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a9b11b63f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Create Your Own Custom Image Classifier And Deploy It To Production]]></title>
            <link>https://medium.com/analytics-vidhya/how-to-create-your-own-custom-image-classifier-and-deploy-it-to-production-5506bb26a74b?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/5506bb26a74b</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Sun, 29 Dec 2019 17:00:06 GMT</pubDate>
            <atom:updated>2021-02-11T10:54:51.340Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*z3TacwxbCWG9oXjGh1_zPA.png" /></figure><p>In this article, we are going to make our own custom image classifier. We are going to use the <a href="https://docs.fast.ai/">fast.ai</a> library to train our deep learning model. It is a high-level library build on PyTorch, which allows us to build models using only a few lines of code.</p><p>And we are going to train our model on <a href="https://colab.research.google.com/">Google Colab.</a></p><p>You can check the <a href="https://www.fast.ai/">fast.ai</a> course. It’s really great.</p><p><strong>The steps needed are:</strong></p><ol><li>Download image dataset</li><li>Load and view your data</li><li>Create and train a model</li><li>Interpret the results</li><li>Make a small web-app out of it</li></ol><h3>Downloading Image Data</h3><p>The FastAI library provides functionality for downloading images given a file containing the URLs of these images.</p><p>In order to get the URLs, you will have to go to <a href="https://images.google.com/">Google Images</a> first and you need to search for your category of choice. Make sure the ad-blocker is turned off or you can try it in incognito mode.</p><p>I am going to choose my own categories which are <em>books, watch, earphones, and guitar.</em> You can search for your choices. So what you need to do now is, search your category one by one. Search one and scroll down until enough images are loaded and then execute the following javascript code from the developer console.</p><p>Hit <em>Ctrl+Shift+J</em> to open your developer console.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8c49a9137f601539f07856496f33f0c/href">https://medium.com/media/d8c49a9137f601539f07856496f33f0c/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IzGl-wCBx6lmFoBDFrKDoQ.png" /></figure><p>First copy these lines and paste it in your console and press enter. The links of the image file will get downloaded as a file named <em>download</em>. Then you need to rename the file to <em>book.txt</em> or name according to your own preference. Then do repeat the above steps for more classes.</p><h3>Load and view your data</h3><p>Open my <a href="https://colab.research.google.com/drive/1k7ZlyU1Ad84x_KxiJZj_hLKdpcU8gy8n">Google Colab</a> notebook for our further steps. After opening the notebook. Run the first two cells.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11a76c26ebdb5abb69cad0c7e5b00602/href">https://medium.com/media/11a76c26ebdb5abb69cad0c7e5b00602/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/531/1*6HdAhcopp46YHZ9M5t3ioA.png" /></figure><p>This above code in the notebook is for specifying the path and creating its corresponding folder.</p><p>After executing the above in the notebook, you need to run the cell below in the notebook and upload the corresponding .txt file.</p><pre>from google.colab import files<br>book = files.upload()</pre><p>You need to upload the .txt file for other classes also.</p><p>Now after uploading all the files, You need to execute other cells one by one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*zchBjYPk5oNBjznCZ-KPTA.png" /></figure><p>What this code will do is, it will read those URLs which are in the text files and it will download the images in Colab.</p><p>After downloading those images, we need to remove the images which can’t be opened.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf41f5327b5051feaf959ab1a62772d7/href">https://medium.com/media/bf41f5327b5051feaf959ab1a62772d7/href</a></iframe><p>This code above will do that for us. This<em> verify_images </em>function looks for the images and sees if there’s any problem with it and <em>delete =True </em>will delete it for you. So it’s a really good and easy way to clean our data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f91fd469baf0d6d9c7b230b1d8140adc/href">https://medium.com/media/f91fd469baf0d6d9c7b230b1d8140adc/href</a></iframe><p>The fast.ai library has specific data objects called <em>databunch</em>es. It helps to read our dataset in a proper folder structure and this helps create validation data randomly. You can read more about in their docs <a href="https://docs.fast.ai/">https://docs.fast.ai/</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*QYxY-wOMnYyPWXBq3MPY3Q.png" /></figure><p>We have got a data bunch. So you can look inside at the data.classes and you&#39;ll see these are the folders that we created. So it knows that the classes (by classes, we mean all the possible labels) are book, watch, earphones &amp; guitar.</p><p>Then you need to run some code below in the notebook. I have commented on the codes so that it would be easier for someone to look into and understand.</p><h3>Creating a model and initial training</h3><p>The fastai library lets you create models in a few lines of code and train a CNN model by <a href="https://docs.fast.ai/vision.learner.html#create_cnn"><em>create_l</em></a><em>earner </em>method which they provide.</p><pre>learn = cnn_learner(data, models.resnet34, metrics=error_rate)</pre><p>This cnn_learner method needs two arguments, the data, and the architecture and it supports many other parameters also.</p><p>The created model uses the <a href="https://arxiv.org/pdf/1512.03385.pdf">resnet34 architecture,</a> with weights pretrained on the <a href="http://www.image-net.org/">imagenet dataset</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*rSlsYiarlstqqta0SvSz6g.png" /></figure><p>The <em>fit_one_cycle </em>method<em> </em>is what trains the model. It uses something called the <a href="https://sgugger.github.io/the-1cycle-policy.html"><strong>1 cycle policy</strong>,</a> which basically changes the learning rate over time to achieve better results.</p><p>Then we save the model. It will take not take more than 5 minutes to train the model.</p><h3>Interpretation</h3><p>Lastly, we can use the FastAIs <em>ClassificationInterpretation</em> class to interpret our results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*UA-U9I5KVK01JGSpZ49UBA.png" /></figure><p>After running all the cells and training the model. The model file will get exported and you can download the weight file and use it for inference and for production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*-yPGgpwPP39PdR2FhgFn5g.png" /></figure><p>You can download the weight file by going to the side by menu and browsing through the folder. After clicking the download, it will get download within some minutes.</p><h3>Making a small web-app for our image classifier</h3><p>Clone my Github Repository first by</p><pre>git clone <a href="https://github.com/TheCaffeineDev/Custom-Image-Classifier">https://github.com/TheCaffeineDev/Custom-Image-Classifier</a></pre><p>After cloning go inside the folder by</p><pre>cd Custom-Image-Classifier</pre><p>Then install all the dependencies by</p><pre>pip3 install -r requirements.txt</pre><p>After that, inside there will be one folder called models, you need to put your <em>export.pkl </em>file there. This is the path to the folder.</p><pre>Custom-Image-Classifier/app/models/ </pre><p>After copying the file you can run the server by</p><pre># In Custom-Image-Classifier folder run</pre><pre>python3 app/server.py serve </pre><p>After some seconds it will run on your local server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*d0u2uIT_1FBraGAb0kbuYA.png" /></figure><p>Here are some inference results below of my model : -</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/329/1*b2xHtdW8-AjRv3s-nHlX7w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/1*zoDKLc2eSw_FDaUDAKhM_w.png" /><figcaption>-</figcaption></figure><p>If you have followed the above steps properly you will be able to create your image classification model easily.</p><p>The code covered in this article is available as a <a href="https://github.com/TheCaffeineDev/Custom-Image-Classifier">Github Repository.</a></p><p>If you have any questions, recommendations or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:prabhat1015@gmail.com">mail</a>. Feel free to reach out to me.</p><h4>References :</h4><ol><li><a href="https://course.fast.ai/">https://course.fast.ai/</a></li><li><a href="https://course.fast.ai/deployment_google_app_engine.html">https://course.fast.ai/deployment_google_app_engine.html</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5506bb26a74b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/how-to-create-your-own-custom-image-classifier-and-deploy-it-to-production-5506bb26a74b">How To Create Your Own Custom Image Classifier And Deploy It To Production</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learn how to create your own custom object detector using YoloV3.]]></title>
            <link>https://medium.com/analytics-vidhya/custom-object-detection-with-yolov3-8f72fe8ced79?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/8f72fe8ced79</guid>
            <category><![CDATA[yolov3]]></category>
            <category><![CDATA[object-detection]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Sat, 21 Dec 2019 21:33:22 GMT</pubDate>
            <atom:updated>2021-02-11T10:54:24.256Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Guide To Build Your Own Custom Object Detector Using YoloV3</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*onPymLIQSe60d1ln89w2gA.jpeg" /><figcaption>Object-detection</figcaption></figure><p>In this article, I am going to show you how to create your own custom object detector using YoloV3.</p><p>I am assuming that you already know pretty basics of deep learning computer vision.</p><p>I am going to train my model on my own GPU.</p><p><strong>The steps needed are:</strong></p><ol><li>Gathering data</li><li>Converting to YoloV3 format</li><li>Setting Training Pipeline</li><li>Training model</li><li>Exporting weights file.</li><li>Testing object detector</li></ol><h3>1.Gathering Data</h3><p>We are going to get our data from <a href="https://storage.googleapis.com/openimages/web/visualizer/index.html?set=train&amp;type=detection&amp;c=%2Fm%2F0k5j">OpenImagesV5</a>. It was first introduced in 2016, It’s is a collaborative release comprising about nine million images annotated with labels covering thousands of object categories. The new version is an update on 2018’s <a href="https://storage.googleapis.com/openimages/web/factsfigures_v4.html">Open Images V4</a>.</p><p>Open Image V5 features newly added annotations on image segmentation masks for 2.8 million objects in 350 categories. Unlike bounding-boxes that only identify the general area in which an object is located, these image segmentation masks trace the outline of the target object, characterizing its spatial extent with a higher level of detail.</p><p>I am going to choose 4 classes of my choice from OpenImagesV5 that is <em>Backpack, Watch, Book, and Headphones</em>. You can choose as many as you want.</p><p>We will have to download those images first. There’s an amazing <a href="https://github.com/pythonlessons/OIDv4_ToolKit">toolkit from GitHub</a> with a full explanation of how to use it. This toolkit will help us to download the images. The toolkit installation is easy. <a href="https://pylessons.com/YOLOv3-introduction/">This guy</a> has written a tutorial to detect custom objects using yolov3 using Keras. You can check it out, he has explained all the steps. In my post, I am going to use PyTorch and will try to simplify things as much as possible.</p><ol><li><strong>Downloading the toolkit</strong></li></ol><p>First, open up your terminal and clone the toolkit repository</p><pre>git clone <a href="https://github.com/pythonlessons/OIDv4_ToolKit.git">https://github.com/pythonlessons/OIDv4_ToolKit.git</a></pre><p>Then go the folder by</p><pre>cd OIDv4_ToolKit</pre><p>Install all the dependencies by</p><pre>pip3 install -r requirements.txt</pre><p><strong>2. Using toolkit</strong> <strong>&amp; Downloading the data</strong></p><p>First of all how to check if we can download the appropriate image class we need? You need to go to the OIDv5 homepage, click on explore and in a search tab try to find your desired class. In my example, I will search for “Watch”, “Backpack”, “Book”, and “Headphones”. To download all of them you can simply run</p><pre>python3 main.py downloader --classes Watch Backpack Book Headphones --type_csv train --multiclasses 1 --limit 600</pre><p>With this command, 600 training images for each class will get downloaded. If you are using for the first time it will ask you to download (train-annotations-bbox or test-annotations-bbox) CSV file. Type Y, while running the command so that it would get downloaded.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*v6VwiQJ70zIWgKWt_VUViQ.png" /></figure><p>You need to wait for some time to finish the download.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*Y95aRGCablwt0NGEy8Od2A.png" /></figure><p>This is the folder structure :</p><pre>main_folder<br>│   - main.py<br>│   - oid_to_pascal_voc_xml.py<br>│   ..<br>└─── OID<br>    │<br>    └─── csv_folder<br>    │   │<br>    │   └─── class-descriptions-boxable.csv<br>    │   │<br>    │   └─── test-annotations-bbox.csv<br>    │   │<br>    │   └─── train-annotations-bbox.csv<br>    └─── OID<br>        │<br>        └─── Dataset<br>            │<br>            └─── train<br>                │<br>                └─── Watch_Backpack_Book_Headphones0</pre><h3>3. Converting to YoloV3 format</h3><p>Open oid_to_pascal_voc_xml.py and edit line no. 9 and put</p><pre>XML_DIR = &#39;TO_PASCAL_XML&#39;</pre><p>Now you need to run</p><pre>python3 oid_to_pascal_voc_xml.py</pre><p>to generate the object co-ordinates in XML formats. After running this above file, you will get object label files in an XML format in the TO_PASCAL_XML folder.</p><p>Again we need to convert it into YoloV3 format.</p><p>To train a Yolo model there are requirements how an annotation file should be made:</p><ul><li>One row for one image;</li><li>Row format: image_file_path box1 box2 … boxN;</li><li>Box format: x_min,y_min,x_max,y_max,class_id (no space).</li></ul><p>Here is an example:</p><ul><li>path/to/img1.jpg 50,100,150,200,0 30,50,200,120,3</li><li>path/to/img2.jpg 120,300,250,600,4 …</li></ul><p>Now you will have to do some little bit of work with these. Have patience, it’s going to take some manual work now.</p><p>I will use the code of <a href="https://github.com/ultralytics/yolov3">Ultralytics</a> to train our YoloV3 model. Make sure to check their repository, it’s great. I have made some changes in the folder structure and all for our own model. Now clone my repository for the next steps.</p><pre>git clone <a href="https://github.com/TheCaffeineDev/YoloV3-Custom-Object-Detection.git">https://github.com/TheCaffeineDev/YoloV3-Custom-Object-Detection.git</a></pre><p>This is the folder structure</p><pre>main_folder<br>│   - detect.py<br>│   - models.py<br>│   - train.py<br>│  - test.py<br>│  - requirements.txt<br>│  ...<br>└─── cfg<br>└─── conversion<br>    │<br>    └─── output<br>    └─── xmls<br>    │  - classes.txt<br>    │  - xmltotxt.py<br>    │  ...<br>└─── data<br>└─── training<br>│ │<br>│ └─── images<br>│ └─── labels<br>│ - object.names<br>│ - trainer.data<br>│ - yolov3.cfg<br>│ - train_test.py<br>│<br>└─── utils<br>└─── weights</pre><p>Now inside the main folder and run this in your terminal to install all the required dependencies.</p><pre>pip3 install -r requirements.txt</pre><p>Then you have to go back to that OIDV4_Toolkit folder OIDv4_ToolKit/OID/Dataset/train/Watch_Backpack_Book_Headphones/ . Here you need to copy all the images into the following folder in our cloned repository/YoloV3-Custom-Object-Detection/training/images And</p><p>you need to copy all the XML files which are in the folder to /YoloV3-Custom-Object-Detection/conversion/xmls folder. Don&#39;t get confused about these above steps.</p><p>After copying go to the conversion folder. You need to edit the classes.txt file and put your own classes there.</p><p>It looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*GGorOPsvT93cLvjIil-OQA.png" /></figure><p>Now in conversion folder run</p><pre># In YoloV3-Custom-Object-Detection/conversion folder<br>python3 xmltotxt.py -xml xmls -out output</pre><p>This will generate all the label files in the yolov3 format inside the output folder.</p><h3>3. Setting Training Pipeline</h3><p>Now you have to copy all the .txt files which are in YoloV3-Custom-Object-Detection/conversion/output folder to YoloV3-Custom-Object-Detection/training/labels folder.</p><p>Now go to the YoloV3-Custom-Object-Detection/training folder. Now we will edit some files here. Now open that object.names file and edit it out with your own classes.</p><p>Now in</p><pre># In YoloV3-Custom-Object-Detection/training folder<br>python3 train_test.py</pre><p>This above file will generate train.txt and test.txt. You can open and check the file for more details. Basically we are splitting 70% &amp; 30% for training and testing respectively.</p><p>Then open trainer.data . It looks like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/958/1*Ad6DAmFRAk56BMaSTyErEQ.png" /></figure><p>You need to put your no. of classes in the first line, train.txt and test.txt path in 2nd and 3rd line, object.names path in the 4th line.</p><p>Now you need to edit the <strong>*.cfg</strong> file. By default each YOLO layer has 255 outputs: 85 outputs per anchor [4 box coordinates + 1 object confidence + 80 class confidences], times 3 anchors.</p><p>In our case we are using only four classes, then we need to edit the filter. You can reduce filters to filters=[4 + 1 + n] * 3, where n is your class count. This modification should be made to the layer preceding each of the 3 YOLO layers. Also, modify classes=80 to classes=n in each YOLO layer, where n is your class count.</p><p>In our case in yolov3.cfg file I changed the batch size &amp; subdivisions which is in line no. <strong>6</strong> and <strong>7</strong>. Then line no <strong>610</strong> (classes=4) and <strong>603</strong> (filters=27), then line no. <strong>689</strong> &amp; <strong>696</strong>, lastly line no. <strong>776</strong> &amp; <strong>783</strong>. If you are using tiny-yolo, line no. will be different.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*ad73ZBRiRep_M6KnUVpxJA.png" /></figure><p>Now we are ready to train our yolov3 model.</p><h3>4. Training Model</h3><p>Below is my desktop specification in which I am going to train my model.</p><ul><li><strong>GPU:</strong> NVIDIA GeForce RTX <em>2080 SUPER</em> (8GB)</li><li><strong>RAM:</strong> 16GB DDR4</li><li>OS: Ubuntu 18.04</li></ul><p>Now in YoloV3-Custom-Object-Detection folder open terminal and run</p><pre># In YoloV3-Custom-Object-Detection run</pre><pre>python3 train.py --epochs 110 --data training/trainer.data --cfg training/yolov3.cfg --batch 16 --accum 1</pre><p>There are optional arguments that are there, you can check-in train.py file.</p><p>Also, You can <strong>Update hyperparameters</strong> also such as LR, LR scheduler, optimizer, augmentation settings, multi_scale settings, etc in train.py for your particular task. I would recommend you start with all-default settings first updating anything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*UCUL67zP_QKAiL1QVKwKdg.png" /><figcaption>Model Training</figcaption></figure><p>To run tensorboard while training, open another terminal and run</p><pre># In YoloV3-Custom-Object-Detection do<br>tensorboard --logdir=runs</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z4fVasaZVg-8fKBvGrQhQA.png" /><figcaption>tensorboard</figcaption></figure><p>It took me 2 hours to train four objects with 110 epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HJ2LkwSQoMNMga2s02lppA.png" /><figcaption>Results</figcaption></figure><p>If you follow the above steps, you will be able to train your own model properly.</p><h3>5. Exporting weights file.</h3><p>After training the model, we can get the weights file in the weights folder. It’s in PyTorch model format. We need to convert it to darknet format to test it out. Let&#39;s convert it first. Open up the terminal and run</p><pre># convert cfg/pytorch model to darknet weights<br># In YoloV3-Custom-Object-Detection<br>python3  -c &quot;from models import *; convert(&#39;training/yolov3.cfg&#39;, &#39;weights/best.pt&#39;)&quot;<br>​</pre><p>You will get this message after conversion.</p><pre>Success: converted &#39;weights/yolov3-spp.pt&#39; to &#39;converted.weights&#39;</pre><p>Now we will test our object detector. You will have the converted.weights in YoloV3-Custom-Object-Detection folder.</p><h3>6. Testing Object Detector</h3><p>Now to run inference, open up your terminal and run</p><pre># In YoloV3-Custom-Object-Detection - for webcam<br>python3 detect.py --source 0 --weights converted.weights --cfg training/yolov3.cfg --names training/object.names --img-size 416</pre><p>Check detect.py for details.</p><ul><li>Image: --source file.jpg</li><li>Video: --source file.mp4</li><li>Directory: --source dir/</li><li>Webcam: --source 0</li><li>RTSP stream: --source rtsp://192.168.0.1/rtp/470011235daa</li></ul><p>Some of the image inferences below you can see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j9bQAdEsreFZdeANP1SBjA.jpeg" /><figcaption>Inference</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*d0SYwqCVEvakEHKHR_HaOA.jpeg" /></figure><p>Due to the GitHub file size limit, I couldn’t upload it to GitHub. If you want to test my weight file, you can get it by</p><pre>wget <a href="https://www.dropbox.com/s/tou8f0xvs8wdt8a/converted.weights?dl=0">https://www.dropbox.com/s/903xy5ba5htitga/converted.weights</a></pre><p>Link to the repository: <a href="https://github.com/TheCaffeineDev/YoloV3-Custom-Object-Detection">https://github.com/TheCaffeineDev/YoloV3-Custom-Object-Detection</a></p><p>If you want to label your own custom images you can use <a href="https://github.com/tzutalin/labelImg">LabelImg</a>.The process of training and all is almost the same.</p><p>If you have any questions, recommendations, or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my <a href="/service/mailto:prabhat1015@gmail.com">mail</a>. Feel free to reach out to me.</p><p><em>Originally posted in my </em><a href="https://thecodemonk.in/software/object-detection-yolov3"><em>blog</em></a><em>.</em></p><p><strong><em>“Thank You”</em></strong></p><h3>References and Credit</h3><ol><li><a href="https://medium.com/syncedreview/google-releases-open-image-v5-launches-open-image-challenge-91fa802f0edf">https://medium.com/syncedreview/google-releases-open-image-v5-launches-open-image-challenge-91fa802f0edf</a></li><li><a href="https://github.com/ultralytics/yolov3">https://github.com/ultralytics/yolov3</a></li><li><a href="https://pylessons.com/">https://pylessons.com/</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8f72fe8ced79" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/custom-object-detection-with-yolov3-8f72fe8ced79">Learn how to create your own custom object detector using YoloV3.</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Custom Object Detection With Tensorflow Using Google Colab]]></title>
            <link>https://medium.com/analytics-vidhya/custom-object-detection-with-tensorflow-using-google-colab-7cbc484f83d7?source=rss-e644b56a185d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7cbc484f83d7</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[opencv]]></category>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Prabhat Kumar Sahu]]></dc:creator>
            <pubDate>Mon, 09 Dec 2019 07:01:31 GMT</pubDate>
            <atom:updated>2022-11-25T06:38:18.339Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*o8Khjem_Lvbe0p6a" /><figcaption>Photo by <a href="https://unsplash.com/@ja_ma?utm_source=medium&amp;utm_medium=referral">ja ma</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this blog post, we are going to build a custom object detector using Tensorflow Object Detection API. I will choose the detection of apple fruit. But you can choose any images you want to detect your own custom objects.</p><p><strong>The steps needed are:</strong></p><ol><li>Installation</li><li>Gathering data</li><li>Labeling data</li><li>Generating TFRecords for training</li><li>Configuring training</li><li>Training model</li><li>Exporting inference graph</li><li>Testing object detector</li></ol><h3>Installation</h3><p>Make sure you have Python 3.6 or higher version of it. I have done this in my Ubuntu 18.04 machine. If you are using Windows then the process is a little bit different.</p><p>If you already have <strong><em>pip</em></strong> then you are good to go. Otherwise, install <strong><em>pip3</em></strong> by</p><pre>sudo apt install python3-pip</pre><h3>1.1 Tensorflow</h3><p>Then Install Tensorflow using the following command:</p><pre>pip3 install tensorflow</pre><p>If you have a GPU that you can use with Tensorflow:</p><pre>pip install tensorflow-gpu</pre><p>The remaining libraries can be installed on Ubuntu 18.04 using via apt-get:</p><h3>1.2 Other dependencies</h3><pre>sudo apt-get install protobuf-compiler python3-pil python3-lxml python3-tk git</pre><p>Others</p><pre>pip3 install pillow Cython lxml jupyter matplotlib contextlib2</pre><p>Make sure Cython is installed by now and then do</p><pre>pip3 install pycocotools</pre><h3>1.3 Clone the Tensorflow models repository</h3><p>Clone the TensorFlow models repository:</p><pre>git clone <a href="https://github.com/tensorflow/models.git">https://github.com/tensorflow/models.git</a></pre><p><strong>From this point on, this directory will be referred to as the</strong> models <strong>directory.</strong></p><h3>1.4 Setting up the environment</h3><p>After cloning the tf models repo, now go to the research folder</p><pre>cd models/research</pre><p><strong>Protobuf Compilation</strong></p><p>The Tensorflow Object Detection API uses Protobufs to configure model and training parameters. Before the framework can be used, the Protobuf libraries must be compiled. This should be done by running the following command from the tensorflow/models/research/ directory:</p><pre># From tensorflow/models/research/<br>protoc object_detection/protos/*.proto --python_out=.</pre><p><strong>Add Libraries to PYTHONPATH</strong></p><p>When running locally, the tensorflow/models/research/ and slim directories should be appended to PYTHONPATH. This can be done by running the following from tensorflow/models/research/:</p><pre># From tensorflow/models/research/<br>export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim</pre><p><em>Note:</em> This command needs to run from every new terminal you start. If you wish to avoid running this manually, you can add it as a new line to the end of your ~/.bashrc file, replacing pwd with the absolute path of tensorflow/models/research on your system.</p><p><strong>Object Detection Installation</strong></p><pre># From tensorflow/models/research/<br>sudo python3 setup.py install</pre><p><strong>Testing the Installation</strong></p><p>You can test that you have correctly installed the Tensorflow Object Detection API by running the following command:</p><pre>python3 object_detection/builders/model_builder_test.py</pre><p>If the result looks like the following, you’re ready to proceed to the next steps!</p><pre>...............----------------------------------------------------------------------<br><br>Ran 15 tests in 0.120sOK</pre><h3>Gathering data</h3><p><strong>2.1</strong> Open your google chrome browser and install an extension called <a href="https://chrome.google.com/webstore/detail/download-all-images/ifipmflagepipjokmbdecpmjbibjnakm?hl=en">Download All Images</a></p><p><strong>2.2</strong>. Now search your desired image of choice in google images, in my case, it’s “Apple”. Now click the “ “download all images” extension button which will be in your top-right side of the browser. You will get a zip file containing your images. Then extract it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*miaBZ6zD1i_JH3iCDvehYQ.png" /><figcaption>Apple Folder</figcaption></figure><h3>Labeling data</h3><p>Open up your terminal and install <em>LabelImg</em> by</p><pre>pip3 install labelImg</pre><p>LabelImg is a graphical image annotation tool.</p><p>After installing labelImg open it by typing</p><pre>labelImg</pre><p>in your terminal</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OiCJO87VmS_dOkNiDPy82w.gif" /></figure><p>And follow the above like what I am doing. And do this for all the pictures. What it’s doing is, it’s generating one XML file containing the object co-ordinates with its label.</p><p>I labelled around 100 images.</p><p>Now clone my repository</p><pre>git clone https://github.com/TheCaffeineDev/Tensorflow-Object-Detection-API-With-Custom-Dataset.git</pre><p>After cloning go inside the directory.</p><p>Your directory structure should look like this:</p><pre>├── ...<br>    ├── data                 <br>    ├── images<br>    │   ├── train<br>    │   ├── test<br>    ├── utils<br>    │	├── label_map_util.py<br>    │	├── visualization_utils.py<br>    ├── generate_tfrecord.py<br>    └── object-detection.pbtxt<br>    └── transform_image_resolution.py<br>    └── xml_csv.py<br>    └── webcam_inference.py</pre><h3>Generating TFRecords for training</h3><p>Now copy 70% of your image file to train folder <em>images/train</em> and rest 30% to your test folder.</p><p>With the images labelled, we need to create TFRecords that can be served as input data for the training of the object detector. In order to create the TFRecords, we will use two scripts from <a href="https://github.com/datitran/raccoon_dataset">Dat Tran’s raccoon detector</a>. Namely the <em>xml_to_csv.py</em> and <em>generate_tfrecord.py</em> files.</p><pre>├── ...<br>├── data                 <br>├── images<br>│   ├── train<br>│		├── image1.jpg<br>│		├── image1.xml ..<br>│   ├── test<br>│		├── image5.jpg<br>│		├── image5.xml ..<br>├── generate_tfrecord.py<br>└── object-detection.pbtxt<br>└── transform_image_resolution.py<br>└── xml_csv.py<br>└── webcam_inference.py </pre><p>Now in that folder, we can transform our XML files to train_label.csv and test_label.csv by opening the command line and typing:</p><pre>python xml_to_csv.py</pre><p>These create two files in the data directory. One called <em>test_labels.csv</em> and another one called <em>train_labels.csv</em>.</p><p>Before we can transform the newly created files to TFRecords we need to change a few lines in the <em>generate_tfrecords.py</em> file.</p><p>From:</p><pre># TO-DO replace this with label map , replace with your own classes<br>def class_text_to_int(row_label):<br>    if row_label == &#39;apple&#39;:<br>        return 1<br>    else:<br>        return 0</pre><p>If you have multiple classes then,</p><pre># TO-DO replace this with label map<br>def class_text_to_int(row_label):<br>    if row_label == &#39;apple&#39;:<br>        return 1<br>    elif row_label == &#39;banana&#39;:<br>        return 2<br>    elif row_label == &#39;orange&#39;:<br>        return 3<br>    else:<br>        return None</pre><p>Now you can generate the TFRecords by typing:</p><pre>python3 generate_tfrecord.py --csv_input=data/train_labels.csv  --output_path=train.record --image_dir=images/train</pre><pre>python3 generate_tfrecord.py --csv_input=data/test_labels.csv  --output_path=test.record --image_dir=images/test</pre><p>These two commands generate a <em>train.record</em> and a <em>test.record</em> file which can be used to train our object detector.</p><h3>Configuring training</h3><p>The last thing we need to do before training is to create a label map and a training configuration file.</p><h3>Creating a label map</h3><p>The label map maps an id to a name. I have already created a label map file for my training. It looks like this:</p><p>Edit <em>object-detection.pbtxt</em></p><pre>item{<br>  id:1<br>  name:&quot;apple&quot;<br><br>}</pre><p>If you are using multiple classes follow this pattern.</p><pre>item {<br>    id: 1<br>    name: &#39;apple&#39;<br>}<br>item {<br>    id: 2<br>    name: &#39;banana&#39;<br>}<br>item {<br>    id: 3<br>    name: &#39;orange&#39;<br>}<br>item {<br>    id: 4<br>    name: &#39;etc&#39;<br>}</pre><p>The id number of each item should match the id of specified in the <em>generate_tfrecord.py</em> file.</p><h3>Creating a training configuration</h3><p>We are going to train our model in Google Colab.</p><p>Follow this link below. I have documented it properly.</p><p><a href="https://colab.research.google.com/drive/1o7JB0pWanEMn6qnRnEXphu0T4YbuKLL2">Open In Colab</a></p><p>I am using “SSD_MOBILENET_V2” for training and with the batch size of 4. You can change the number of steps, which pre-trained model to use &amp; the batch and size.</p><p>Then you need to run the cells below. There will be one Upload TF Record heading. Below that you need to upload your generated <em>train.record</em>, <em>test.record</em> &amp; <em>object-detection.pbtxt</em> file.</p><h3>Training model</h3><p>Now after uploading all those files, run all the cells below. It will get trained.</p><h3>Exporting inference graph</h3><p>If you have run all the cells, then, at last, a file named <em>frozen_inference_graph.pb</em> file will get downloaded.</p><h3>Testing Object Detector</h3><p>Now copy that <em>frozen_inference_graph.pb</em> file into my GitHub cloned folder. Then you need to edit some things out in that <em>webcam_inference.py</em> to test your own object detector. Open that file and go through the code. I have mentioned the lines you need to change. You can pretty much do that.</p><p>If you have followed all of the above steps properly then you will be able to test your model via webcam.</p><h3>Conclusion</h3><p>The Tensorflow Object Detection API allows you to create your own object detector using the transfer learning technique.</p><p>Here’s the link to grab the code.</p><p><a href="https://github.com/TheCaffeineDev/Tensorflow-Object-Detection-API-With-Custom-Dataset">Github Repo</a></p><p>If you have any questions, recommendations or critiques, I can be reached via <a href="https://twitter.com/thecaffeinedev">Twitter</a> or via my mail.</p><p><em>Originally posted in my </em><a href="https://thecodemonk.in/software/object-detection"><em>blog</em></a><em>.</em></p><p>Thank You</p><h3>References</h3><ol><li><a href="https://github.com/tensorflow/models/tree/master/research/object_detection">https://github.com/tensorflow/models/tree/master/research/object_detection</a></li><li><a href="https://medium.com/@WuStangDan/step-by-step-tensorflow-object-detection-api-tutorial-part-1-selecting-a-model-a02b6aabe39e">https://medium.com/@WuStangDan/step-by-step-tensorflow-object-detection-api-tutorial-part-1-selecting-a-model-a02b6aabe39e</a></li><li><a href="https://pythonprogramming.net/introduction-use-tensorflow-object-detection-api-tutorial/">https://pythonprogramming.net/introduction-use-tensorflow-object-detection-api-tutorial/</a></li><li><a href="https://towardsdatascience.com/creating-your-own-object-detector-ad69dda69c85">https://towardsdatascience.com/creating-your-own-object-detector-ad69dda69c85</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7cbc484f83d7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/custom-object-detection-with-tensorflow-using-google-colab-7cbc484f83d7">Custom Object Detection With Tensorflow Using Google Colab</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>