<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Prathamesh Sarang on Medium]]></title>
        <description><![CDATA[Stories by Prathamesh Sarang on Medium]]></description>
        <link>https://medium.com/@prato_s?source=rss-9f60702e8437------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*tou-3XlcGW6Uv4xLCigrxg.jpeg</url>
            <title>Stories by Prathamesh Sarang on Medium</title>
            <link>https://medium.com/@prato_s?source=rss-9f60702e8437------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:51:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@prato_s/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Docker for DataScience — Part 1: As a Data Scientist or Data Engineer, do you need to start using…]]></title>
            <link>https://medium.com/@prato_s/docker-for-datascience-part-1-as-a-data-scientist-or-data-engineer-do-you-need-to-start-using-d2c378608f95?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2c378608f95</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Mon, 16 Apr 2018 17:30:26 GMT</pubDate>
            <atom:updated>2018-04-27T16:19:51.421Z</atom:updated>
            <content:encoded><![CDATA[<h3>Docker for DataScience — Part 1: As a Data Scientist or Data Engineer, do you need to start using Docker?</h3><blockquote><strong>If you are a Data Scientist, a Data Analyst or a Data Engineer, this post is about helping you decide if you need to start using Docker.</strong></blockquote><p>Automation of Data Science environments, and bringing the development and production environments for Data Science closer to each other are becoming a first-class concerns with every passing day. Docker has been advocated as an important solution to a wide variety of Data Engineering problems like these.</p><p>You might have heard about Docker and want to give it a try. You might even have tried it for a week and then given up in frustration! After reading through this post, you’ll</p><ul><li>Understand what Docker is, and how it can be useful,</li><li>Understand where Docker can help you in your Data Science workflow,</li><li>Determine if Docker is a good fit for the current problems you have in your Data Science workflow.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*r8Re09WenAKyCTV4ZkFrpg.jpeg" /><figcaption><strong><em>Experience with Docker over time. Hang in there!</em></strong></figcaption></figure><p>Here is how this post is organized:</p><ol><li>Challenges we face as Data Scientists and Data Engineers</li><li>What is Docker? What can it do?</li><li>Docker Application Scenarios</li><li>Challenges in using Docker for Data Science (and how to overcome them)</li><li>Next Steps</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fa1b72f%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fa1b72f%2F&amp;image=https%3A%2F%2Fe.enpose.co%2F%3Fkey%3DdRXnS9Gplk%26w%3D700%26h%3D425%26url%3Dhttps%253A%252F%252Fupscri.be%252Fa1b72f%252F%253Fenpose&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/40e28dee761d48b90335b50b32e46255/href">https://medium.com/media/40e28dee761d48b90335b50b32e46255/href</a></iframe><h3>Our challenges as Data Scientists and Data Engineers</h3><p>Data Science is about ideas, experimentation and sharing insights. Data Science folks are great at all those, but they seem to hit a snag once it involves getting it out to the people outside of the team.</p><p>Getting the results from the Data Scientist’s desk to the business, where they can use it to make decisions, is of the utmost priority. The impetus is more on getting out experiments right and getting them out fast, whatever it takes to achieve that. Over time, this leads to technical debt (and dependency hell) that might make our lives difficult if we revisit the experiments.</p><h3>What is Docker? What can it do?</h3><p>Docker is awesome. People routinely use it for:</p><ul><li>automating, sharing and reproducing experiments,</li><li>packaging and deploying Data Science applications,</li><li>creating easy to use Data Science sandboxes for on-boarding and experiments, and</li><li>large scale data analysis and machine learning at scale in cloud environments.</li></ul><h3>But what is Docker? What can it do for Data Engineering?</h3><blockquote>Docker<em> is an open platform for developers and sysadmins to build, ship, and run distributed applications, whether on laptops, data-center VMs, or the cloud. (Official Doc)</em></blockquote><ul><li>Docker uses containers to make it easier to create, deploy, and run applications.</li><li>Containers enable a programmer or a data engineer to isolate and package an application with all the dependancies it needs (files, libraries, etc).</li><li>The whole thing ships as one package, which can be run on any other machine running on the Linux OS, reproducing the packaged environment exactly irrespective of how the target machine is configured.</li></ul><p>Importantly, Docker has seen an equal following in the start-up and larger enterprise space, validating its usefulness across the spectrum of engineering tasks.</p><ul><li>NetFlix runs a fleet of containers in their operations infrastructure (<a href="https://medium.com/netflix-techblog/updates-on-netflixs-container-management-platform-a91738360bd8">read more</a>).</li><li>Majority of the cloud providers have some form of support or platform based applications to run containers (more precisely Docker).</li><li>To give an example, AWS provides <a href="https://aws.amazon.com/ecs/">AWS Container Service</a> that helps to create a high performant, scalable and easy to orchestrate native container service.</li></ul><h3>How hard is it to learn Docker</h3><p>The Docker learning curve is manageable, but yet for folks in Data Science with less exposure to the technical side of things might find it steep. We’ll talk about how to get started in your Docker journey, but let’s first look at usecases where Docker really shines.</p><h3>1. Do you need to share and reproduce your analysis and experiments?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*JHDYKiL4jZR7Ie0Bpb2AmQ.png" /></figure><p>One of the biggest pain points in Data Science is sharing and reproducing experiments. Challenges involved in this are:</p><p><strong>Different Operating Systems</strong></p><ul><li>Everyone makes a choice and these choices breaks installations in lot of Data Science libraries.</li><li>A library that installs smoothly in Linux based systems might not be as easy in Windows.</li></ul><p><strong>Versioning</strong></p><ul><li>Different python/R versions, different library versions can wreck havoc while sharing analysis.</li><li>There have been efforts to streamline this process, yet the aforementioned Operating System issue is still a big factor.</li></ul><p><strong>Lack of process</strong></p><ul><li>A lot of times in smaller teams and even at enterprise level, the efforts in creating processes while sharing these experiments are either non-existent or are aligned to the regular reporting work that is a manual process.</li><li>Data Science experiments are creative and non-linear, which makes it difficult to neatly fit them into processes that already exist in a system.</li></ul><p>Docker helps in all the above. Since a Docker Image is immutable, the issue of conflicting operating system gets resolved. The versions of operating system and libraries stays frozen (unless updated) and we can set up workflows that integrate Docker in each person involved in the Data Science process (all the technical folks).</p><h3>2. Is building proof-of-concept (or production-level) Dashboards, Web APIs and backend Cron-jobs a part of your job?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*LQf18c_pKBzrkyo_tamVxg.jpeg" /></figure><p>Data Science applications can come in many types:</p><ul><li>Dashboards using RShiny/Dash/Bokeh</li><li>Web APIs created to serve Machine Learning models</li><li>Scripts that run regularly (batch processes) using Machine Learning models</li></ul><p>Sharing Data Science applications to end users is a big task. You don’t want to share it publicly, only to a specific subset. Dashboards for Proof of Concepts (POCs) are a tough task, sharing RShiny or Python code directly doesn’t really cut if the user is non-technical. It defeats the entire purpose of a web based <em>user interface that’s easy to use</em>. Docker and its eco-system can help Data Scientists to at least try to publish Data Science applications for a limited user-base.</p><h3>3. Do you need to create Data Science sandboxes for on-boarding and experiments?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*DuENXdxYvPR9n_7JgiMPqg.png" /></figure><p>Onboarding is a key concern for a lot of big organisations. Corporate firewalls, ServiceNow requests to install Anaconda are some technical hurdles that eat out on the productive time.</p><p>Docker based images can help new people get up and running quickly on their laptops/desktops with just a few commands. Sandboxes for quick feasibility analysis can also be resolved with Docker. If there are new things the team wants to try without the headache of installation, all we need is a Docker image for that.</p><h3>4. Would you like to improve your deployment process for Data Science applications?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*naGEUgXHHGBE-QYcuVKH4A.jpeg" /></figure><p>Deployments are a big headache. It’s common to have a huge information asymmetry between Data Science groups and the DevOps group on how a Machine Learning or Data Science application should be integrated in a system.</p><p>Both the groups work in isolation and often the regular software engineering deployments are not effective with the unique problems associated with Machine Learning applications. Docker can help to bridge the issues concerning the above, if the Data Science groups are enabled to do small scale deployments or a hybrid team that works in tandem with Data Science groups, enabled by Docker.</p><h3>5. Do you want to make it easier to carry out large scale data analysis and machine learning training in cloud environments</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*CDaEs24lH97choQeKdQBMA.png" /></figure><p>With the increasing complexities in data analysis (mostly due to volume) and machine learning (training models), more computing power is needed. Majority of the Cloud providers have provisions to access their compute power via their platform.</p><p>In these cases, time is money and tools like Docker along with Chef/ Puppet/ Terraform can help in saving time in setting up and managing cloud infrastructures.</p><h3>Challenges in Using Docker for Data Science</h3><p>1. Not enough resources for Data Science applications using Docker.</p><p>2. New skill that has a steep learning curve.</p><p>3. Getting entire team to change their workflow to integrate Docker.</p><p>Now we have a good understanding of usecases for Docker.</p><p>If you are struggling with some of these problems, it would probably be a good idea to evaluate Docker for you workflow or your Organisation. To learn more about Docker, what it can do, and how to use it practice/production, watch this space. I’ll be blogging more about it in the coming weeks.</p><p>You can also sign up for the newsletter below to stay updated about my upcoming blogposts in this series. If you can’t wait to get started, here are a few links to set you on your way.</p><p><em>Originally published at </em><a href="https://theaijournal.com/2018/04/16/docker-for-datascience-1-as-data-scientist-data-engineer-why-start-using-docker/"><em>theaijournal.com</em></a><em> on April 16, 2018.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Fa1b72f%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Fa1b72f%2F&amp;image=https%3A%2F%2Fe.enpose.co%2F%3Fkey%3DdRXnS9Gplk%26w%3D700%26h%3D425%26url%3Dhttps%253A%252F%252Fupscri.be%252Fa1b72f%252F%253Fenpose&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/40e28dee761d48b90335b50b32e46255/href">https://medium.com/media/40e28dee761d48b90335b50b32e46255/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2c378608f95" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Deep Learning GPU environment]]></title>
            <link>https://becominghuman.ai/setting-up-deep-learning-gpu-environment-5651564ff936?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/5651564ff936</guid>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Tue, 02 May 2017 14:20:41 GMT</pubDate>
            <atom:updated>2017-05-05T18:31:04.549Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MkprgItZpreEMpx2.jpg" /></figure><p><strong>Source: YouTube: </strong><a href="https://www.youtube.com/watch?v=UQAwOx5UxYU"><strong>@HeroTech</strong></a></p><p>Setting up a Deep Learning Environment for a GPU Enabled system is a headache. It is harder compared to the simple CPU setups that take hardly much time. This post is for you, if you have struggled to get things working on Ubuntu.</p><p>(Note: There might be a few changes in getting things working for your system, since this might get very obsolete in a month or few months time)</p><p>You’ll be going step-by-step in how not to sweat it out while getting things working. GPUs don’t come cheap and it is obvious that no one wants to screw things up while working on the same. Below are the sections that we’ll go through:</p><ul><li><strong>Basic checks for installations needed for CUDA 8.0 and cuDNN.</strong></li><li><strong>Installing the NVIDIA Drivers along with CUDA 8.0.</strong></li><li><strong>Installing cuDNN.</strong></li><li><strong>Installing Python using MiniConda.</strong></li><li><strong>Installing GPU Version of TensorFlow.</strong></li><li><strong>Installing the other required libraries to work with (Keras, Lasagne).</strong></li><li><strong>Checking whether everything works.</strong></li></ul><p>Of all the challenges, <strong><em>NVIDIA Driver installation</em></strong> is the biggest headache involving the most tedious workaround.</p><p>Our machine specs:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/452c745dbc400116c5fb753370957131/href">https://medium.com/media/452c745dbc400116c5fb753370957131/href</a></iframe><p><em>The GPU we have isn’t an NVIDIA one, but from MSI (so be careful while following this blog post if you have an NVIDIA GPU).</em></p><p>Let’s start off:</p><h3>Basic checks for installations needed for CUDA 8.0 and cuDNN</h3><ul><li>Check for the nvidia enabled drivers:</li></ul><p>lspci | grep -i nvidia</p><ul><li>Check for the version of gcc (We need to install version less than 5.0 for some issues with Theano):</li></ul><p>gcc --version</p><ul><li>If version is above 4.9, downgrade the existing version from 5.3 to 4.</li></ul><pre>sudo apt-get install gcc-4.9<br>sudo apt-get install g++-4.9</pre><ul><li>Need to remove the initial installation for gcc:</li></ul><pre>sudo rm -rf /usr/bin/gcc<br>sudo rm -rf /usr/bin/g++</pre><ul><li>Create hard links for the gcc versions:</li></ul><pre>sudo ln -s /usr/bin/gcc-4.9 /usr/bin/gcc<br>sudo ln -s /usr/bin/g++-4.9 /usr/bin/g++</pre><ul><li>Check whether things are working or not:</li></ul><pre>gcc --version<br>g++ --version</pre><h3>Installing the NVIDIA Drivers along with CUDA 8.0</h3><ul><li>You need to download the CUDA Installer kit from <a href="https://developer.nvidia.com/cuda-downloads">this location (For Linux distribution)</a></li><li>Path: Linux --&gt; x86_64 --&gt; Ubuntu --&gt; 16.04 --&gt; runfile(local) --&gt; Base Installer (1.4 GB)</li><li>Create a folder to extract contents of the installer file (Optional)</li></ul><pre>mkdir ~/Downloads/nvidia_installers</pre><pre>cd ~/Downloads</pre><ul><li>Instead of this directly copy-paste, run:</li></ul><p>sudo sh ./cuda_8.0.44_linux.run --extract=~/Downloads/nvidia_installers</p><ul><li>Follow this step only if you have an existing NVIDIA installer in your system, after this is done you will have the open source driver coming in picture.</li><li>In the next few steps we would take care of that:</li></ul><p>sudo apt-get --purge remove nvidia-&lt;version name&gt;</p><ul><li>I’m not sure if the issue: <a href="https://askubuntu.com/questions/760934/graphics-issues-after-while-installing-ubuntu-16-04-16-10-with-nvidia-graphics">Graphics issues after/while installing Ubuntu 16.04/16.10 with NVIDIA graphics</a> got resolved (in 2017). Updating the Graphics Driver from GUI Console didn’t help the cause. So we had to do a work-around for that (follow along).</li><li>Need to create the xorg.config file. You would have it in the location /etc/X11. Not needed for a fresh install, still run it.</li></ul><p>sudo rm /etc/X11/xorg.conf</p><ul><li><strong>Blacklist the Nouveau module:</strong></li></ul><blockquote>To list the available modules in the system: lsmod</blockquote><blockquote>To check for the dependencies of that module: modinfo -F depends nouveau</blockquote><blockquote>To remove without the dependencies, create /etc/modprobe.d/blacklist.conf and add the lines below:</blockquote><blockquote>blacklist nouveau<br>options nouveau modeset = 0</blockquote><ul><li>Download the Nvidia driver for 16.04 (Recommended nvidia-367). Copy everything in the Downloads/nvidia_installers</li><li>Run the commands below</li></ul><p>sudo update-initramfs -u</p><ul><li>Reboot the computer after this to boot into TTY mode:</li></ul><blockquote>Reading for TTY mode: <a href="https://askubuntu.com/questions/481906/what-does-tty-stand-for">What does TTY stand for?</a> &amp; <a href="http://www.linusakesson.net/programming/tty/">TTY Demystified</a></blockquote><blockquote>Reading on how to boot into TTY mode: <a href="http://ubuntuhandbook.org/index.php/2014/01/boot-into-text-console-ubuntu-linux-14-04/">Link</a></blockquote><blockquote>While booting up, press Crtl+Alt+F1</blockquote><blockquote>You should see a black screen terminal (see below):</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*0gSEwu4PekyqdkpxUdtaew.png" /></figure><ul><li>The next step is to: <em>Stop the XServer</em>. Reading on: <a href="https://askubuntu.com/questions/7881/what-is-the-x-server">What is XServer</a>.</li><li>Run the command:</li></ul><p>sudo service lightdm stop</p><ul><li>You have to run the CUDA installation in the TTY for mode: sudo sh ./cuda_8.0.44_linux.run</li><li>The above command installs the driver as well as the tool kit, driver is the one for 16.04. We had downloaded the 16.04 version.</li><li>You will encounter the following questions:</li></ul><blockquote>Scroll through the entire document, there will be a few questions:</blockquote><blockquote>EULA Acceptance? Accept</blockquote><blockquote>CUDA Driver installation? Y</blockquote><blockquote>Do you want to install OpenGL Libraries? N (Y for Nvidia Drivers)</blockquote><blockquote>Do you want to run nvidia-xconfig? Y</blockquote><blockquote>Install Cuda 8.0 toolkit? Y</blockquote><blockquote>Enter tool kit location? Enter a new one or default</blockquote><blockquote>Do you want to install a symbolic link at ? Y</blockquote><blockquote>Install the CUDA Samples? Y</blockquote><blockquote>Enter CUDA Sample location: Enter the path of Cuda directory/samples</blockquote><blockquote>Missing recommended libraries: Upto you.</blockquote><ul><li>Export the paths: sudo nano .bashrc and put the export comments (below) at the end of the file:</li></ul><p>export PATH=/usr/local/cuda-8.0/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH</p><ul><li>Run: source .bashrc</li><li>Check the driver version: cat /proc/driver/nvidia/version</li><li>Should be whatever version you install, our system has:</li></ul><pre>NVRM version: NVIDIA UNIX x86_64 Kernel Module  375.39  Tue Jan 31 20:47:00 PST 2017<br>GCC version:  gcc version 4.9.3 (Ubuntu 4.9.3-13ubuntu2)</pre><ul><li>Check the CUDA Version: nvcc -V</li><li>If it is successful, you should have the output below:</li></ul><pre>nvcc: NVIDIA (R) Cuda compiler driver<br>Copyright (c) 2005-2016 NVIDIA Corporation<br>Built on Sun_Sep__4_22:14:01_CDT_2016<br>Cuda compilation tools, release 8.0, V8.0.44</pre><ul><li>Restart the lightdm service, this will take you back to your GUI console.</li></ul><p>sudo service lightdm start</p><ul><li>Make sure you remember where you saved your CUDA Samples. We saved ours in: /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery</li><li>Fire up the terminal and run the commands below:</li></ul><p>cd /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery</p><ul><li>Run: sudo make. This would compile the files for you, for the program to be run.</li><li>To run the deviceQuery sample, run: ./deviceQuery. It should give you the details of your device (below):</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/26e67e70635f6932e7318d870f6ae569/href">https://medium.com/media/26e67e70635f6932e7318d870f6ae569/href</a></iframe><h3>Installing cuDNN</h3><ul><li>Download the cuDNN zip folder from this location: <a href="https://developer.nvidia.com/rdp/cudnn-download">cuDNN Download</a></li></ul><p>(<em>Note: First time download needs a login and need to create Nvidia Developer account — going through the motions of creating an account, answering the survey and other things and also activating the account after which the downloads would be available for us mortals.</em>)</p><ul><li>Choose “<strong>cuDNN v5.1 Library for Linux</strong>” and follow the steps below:</li></ul><pre>tar xvzf ~/Downloads/cudnn-8.0-linux-x64-v5.1.tgz</pre><pre>cd cuda</pre><pre>sudo cp include/cudnn.h /usr/local/cuda/include/</pre><pre>sudo cp lib64/* /usr/local/cuda/lib64/</pre><ul><li><strong>cuDNN v5.1</strong> should now be available for you.</li></ul><p>Now you are done through with the most difficult part, the next set shouldn’t be taxing!</p><h3>Installing Python using MiniConda</h3><ul><li>It is always better to use Continuum’s Anaconda distribution, but it is best to use <a href="https://conda.io/miniconda.html">Miniconda</a> as the minimal base python version.</li><li>Use any version from <a href="https://conda.io/miniconda.html">this link</a>.</li></ul><pre>wget <a href="https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh">https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh</a></pre><pre>bash Miniconda3-latest-Linux-x86_64.sh</pre><ul><li>A set of questions would follow, answer them appropriately. If interested in reading more and getting a pre-designed Python environment follow <a href="https://github.com/soumendra/python-machinelearning-setup">this link</a></li><li>Test it once on the terminal, to make sure you are running the Miniconda Version.</li><li>Create the python3.5 environment: conda create --name dlgpu python=3.5</li><li>Activate the environment: source activate dlgpu</li></ul><h3>Installing GPU Version of TensorFlow</h3><ul><li>Visit the site: <a href="https://www.tensorflow.org/versions/r0.11/get_started/os_setup.html">OS Setup — TensorFlow</a></li><li>Follow the steps below:</li></ul><pre>export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.0.1-cp35-cp35m-linux_x86_64.whl</pre><ul><li>Update setuptools (through conda to avoid install_path.pth issues)</li></ul><p>conda install setuptools</p><ul><li>Now install TensorFlow GPU using the command:</li></ul><p>pip install --ignore-installed --upgrade $TF_BINARY_URL</p><ul><li>For this issue: <a href="https://github.com/NVIDIA/DIGITS/issues/8">NVIDIA DIGITS Issue</a></li></ul><p>sudo ldconfig /usr/local/cuda/lib64</p><h3>Installing the other required libraries to work with (e.g. Keras, Lasagne)</h3><ul><li>Install other libraries that you’ll need, we’ll be going for keras &amp; lasagne</li></ul><pre>pip install keras<br>pip install lasagne</pre><h3>Checking whether everything works</h3><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ab5a1e458cd7056ccf836cafaa1eabf9/href">https://medium.com/media/ab5a1e458cd7056ccf836cafaa1eabf9/href</a></iframe><p>If you are here, I felt the same as you!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*tTMwjLX-fWF87rNP37Z-zw.gif" /></figure><p>This should get your started with Deep Learning using GPU. Do let me know in the comments if there was any issue during the installation, would try to answer the questions to the best of my knowledge.</p><p><strong>Sources:</strong></p><ol><li><a href="http://www.nvidia.com/object/gpu-accelerated-applications-tensorflow-installation.html">NVIDIA Blog: Installation</a></li><li><a href="http://deeplearning.lipingyang.org/2017/01/18/install-gpu-tensorflow-ubuntu-16-04/">Install GPU TensorFlow Ubuntu 16.04</a></li><li><a href="https://github.com/floydhub/dl-setup">floydhub/dl-setup</a></li><li><a href="http://www.computervisionbytecnalia.com/en/2016/06/deep-learning-development-setup-for-ubuntu-16-04-xenial/">Setup for Ubunutu 16.04</a></li><li><a href="https://alliseesolutions.wordpress.com/2016/09/08/install-gpu-tensorflow-from-sources-w-ubuntu-16-04-and-cuda-8-0-rc/">Installing GPU Tensorflow in Xenial 16.04</a></li><li><a href="http://kislayabhi.github.io/Installing_CUDA_with_Ubuntu/">Older setup</a></li><li><a href="https://www.born2data.com/2016/deeplearning_install-part1.html">Deep Learning Installation</a></li><li><a href="https://github.com/philferriere/dlwin">Deep Learning setup for Windows</a></li><li><a href="http://technozed.com/fix-nvidia-graphical-issues-installing-ubuntu-16-04/">NVIDIA Driver issues</a></li><li><a href="https://linuxconfig.org/how-to-install-the-latest-nvidia-drivers-on-ubuntu-16-04-xenial-xerus">How to chose the driver for Linux</a></li></ol><p><em>Originally published at </em><a href="https://pratos.github.io/setup-tensorflow-gpu-on-ubuntu.html"><em>pratos.github.io</em></a><em>.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*bQlRSzFHJEmF4Q7PyrLgng.gif" /></figure><figure><a href="https://medium.com/becoming-human/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://medium.com/becoming-human/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5651564ff936" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/setting-up-deep-learning-gpu-environment-5651564ff936">Setting up Deep Learning GPU environment</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simplified Docker-ing for Data Science — Part 1]]></title>
            <link>https://becominghuman.ai/docker-for-data-science-part-1-dd41e5ef1d80?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd41e5ef1d80</guid>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[docker-image]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Tue, 25 Apr 2017 08:57:53 GMT</pubDate>
            <atom:updated>2017-07-28T11:23:05.245Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/679/0*Ol_D5wBqZxib_DuD.png" /></figure><p>Source: Docker Homepage</p><h3>The situation</h3><p>As Data Scientist/Data Analysts/Data Science enthusiasts, we are by far a messier version of our Software Engineering counter parts. We like quick, dirty solutions. Code base that sometimes puts a double burst cheese pizza to shame (yum though!).</p><p>We like to experiment, change things on the fly and get the results out as soon as possible…coz deadlines <em>shakes head</em>. All this adds to the mess that already is — dependencies, obscure packages and platform dependent issues. To replicate the same experiment takes more than just writing random.seed(4), and there comes the classic problems:</p><blockquote>“Okay! But that ran on my system”</blockquote><blockquote>“I have sent every code script, data and other things. Strange I haven’t missed any.”</blockquote><blockquote>“Too lazy to install everything from scratch, hate Linux/Windows/Mac”</blockquote><blockquote>“Can’t install the package that you used, can you help me out?”</blockquote><p>What if I told you, as Data Scientist one can easily get all this sorted? Docker is the answer!</p><p>There’ll be a section of readers who would be thinking, <em>“Docker is nuts! Only DevOps folks know it in and out!”</em>. True in some respects, but if there’s a way to standardize our Data Science Workflow and be productive then why not try it out. Knowing just small bit of Docker would be enough as a starting point.</p><h3>What is Docker?</h3><p>According to Docker’s official website:</p><blockquote><em>Docker is the world’s leading software container platform. Developers use Docker to eliminate “works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux and Windows Server apps.</em></blockquote><p>Docker provides us an isolated container, where we can add all the things that we need to for our experiment to run. We could also create a full fledged web application that can be used as a production grade. So right from experiments to production applications, Docker provides a good means to get our Data Science applications out in the real world.</p><p>To learn Docker, we need to understand the terminologies and tools on offer and what we can do with it.</p><p><strong>Docker Terminology:</strong></p><ol><li><strong>Docker Containers:</strong> Small user-level virtualization (isolation) that helps you install, build and run your code/workflow. All the code would be continuosly running in these containers.</li><li><strong>Docker Images:</strong> An image is an inert, immutable, file that’s essentially a snapshot of a container. These are your actual committed containers (ones that have the process running, data stored, ports exposed to be used). Docker images are essentially the stored instances that you can (actually move around).</li><li><strong>Dockerfile:</strong> It is a YAML (almost) based file from which <em>Docker</em> creates an image. It can be thought of as an automated script that has all the steps you want to execute.</li></ol><p>You could be a Data Science enthusiasts who can share his code without documenting any single thing, Dockerfile (more on this later) is self-documenting. All you have to do is create a Docker image, upload it on DockerHub and share it with the world!</p><p>A good resource to know more about <strong>Docker and Containerization technology</strong> as a whole read <a href="https://medium.freecodecamp.com/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b">this blog</a>.</p><p>In this blog post, we’ll tackle very narrow problems of <strong>Starting off with pre-built Docker images</strong> and <strong>Environment packaging</strong>.</p><h3>Docker installation</h3><p>All the examples here are generic, except for the installation of <em>Docker</em>. The official <em>Docker installation</em> documents for Windows and Mac are pretty straight forward. You can start off <em>Docker installation</em> using this <a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04">DigitalOcean blog post</a>.</p><p>(<em>Tip: For any Linux distro related installations, DigitalOcean articles are the real deal! Always search for them.</em>)</p><h3>Docker for instant gratification</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*XEUZUBNYnDnVZhZj.jpg" /></figure><p>Getting the right tool to do things is hard and getting it installed on your system is a tough job sometimes.</p><p>Let’s take an example Python’s cryptography installation on your Windows system, unless you have the right Microsoft authorized C++ compiler the package won’t pip install. After ardorous, time sapping attempts you might get things running. In Ubuntu, installation is just a breeze.</p><p><a href="https://medium.com/becoming-human/tensorflow-serving-by-creating-and-using-docker-images-336ca4de8671">Read my experiences</a> in compiling TensorFlow Serving, that would give you a fair idea that setting up platform to do things, especially obscure things take time.</p><p>Docker solves this, <a href="https://hub.docker.com/">DockerHub</a> has a host of images uploaded by individuals and tech teams alike. Right from <a href="https://hub.docker.com/_/hello-world/">Hello World images</a> to <a href="https://hub.docker.com/r/sergiobuj/cl-jupyter/">LISP images</a>.</p><p>What do you need to do? Just docker pull &lt;username&gt;/&lt;image-name&gt;. It would download the image for you. Try out the following:</p><pre>user@user:~$ docker pull hello-world </pre><pre>Using default tag: latest <br>latest: Pulling from library/hello-world <br>78445dd45222: Pull complete <br>Digest: sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7 <br>Status: Downloaded newer image for hello-world:latest</pre><p>To run the <em>docker image</em> run command as below:</p><p>docker run hello-world</p><p>Should give you output:</p><pre>Hello from Docker! </pre><pre>This message shows that your installation appears to be working correctly. </pre><pre>To generate this message, Docker took the following steps: <br>1. The Docker client contacted the Docker daemon. <br>2. The Docker daemon pulled the &quot;hello-world&quot; image from the Docker Hub. <br>3. The Docker daemon created a new container from that image which runs the executable that produces the output you are currently reading. <br>4. The Docker daemon streamed that output to the Docker client, which sent it to your terminal. </pre><pre>To try something more ambitious, you can run an Ubuntu container with: </pre><pre>$ docker run -it ubuntu bash </pre><pre>Share images, automate workflows, and more with a free Docker ID: https://cloud.docker.com/ </pre><pre>For more examples and ideas, visit: <a href="https://docs.docker.com/engine/userguide/">https://docs.docker.com/engine/userguide/</a></pre><p>Generally the <em>docker files</em> that you would use for Data Science purposes would be much complex with a variety of components to take care of: <em>ports, volumes and many other things.</em></p><p>We’ll look at how to run a full fledged <em>Anaconda</em> jupyter notebook that has all the components pre-installed. (An easier version of <em>conda installation</em>).</p><pre>user@user:~$ docker pull continuumio/anaconda3 </pre><pre>Using default tag: latest <br>latest: Pulling from continuumio/anaconda3 <br>8ad8b3f87b37: Pull complete <br>26e5bbd29116: Pull complete <br>26a23cde1ff7: Pull complete <br>0947a413f98b: Pull complete <br>Digest: sha256:d04148529d340097c08677061e7d08f42e13a51dbdbf7488d92af0a65fd82973 <br>Status: Downloaded newer image for continuumio/anaconda3:latest</pre><p>To log in the docker container, we can run the command below: — docker run -i -t continuumio/anaconda3 /bin/bash</p><p>To run Jupyter notebook in a no-browser mode, run the command below: - docker run -i -t -p 8888:8888 continuumio/anaconda3 /bin/bash -c &quot;/opt/conda/bin/conda install jupyter -y --quiet &amp;&amp; mkdir /opt/notebooks &amp;&amp; /opt/conda/bin/jupyter notebook --notebook-dir=/opt/notebooks --ip=&#39;*&#39; --port=8888 --no-browser&quot;</p><p>Here, let’s discuss what the 2nd command is doing:</p><ul><li>-i is running the image interactively.</li><li>-t is allocate a pseudo-TTY.</li><li>-p is connect/publish the container ports to host. Here localhost:8888 to 8888 of container.</li></ul><p>Downloading jupyter and starting the notebooks in the --no-browser mode.</p><p>A few more that you should know are:</p><ul><li>-d is to run container in the background, without accidently closing it.</li><li>-v is to bind a volume to the container, useful when you want your notebooks and/or data to be reflected in the container from your local system.</li></ul><p>Go to the browser and run <a href="http://localhost:8888/">http://localhost:8888</a> to start running your experiments. Pretty nifty, if you want to start off easily using a ready-made docker container.</p><p>But, all of this doesn’t solve our main problem: <strong>Managing Environments</strong>.</p><h3>Manage Environments and let others use that easily!</h3><p>For my daily work, I create virtual environments for each project using Anaconda distribution. It provides a very easy means to share code along with the environment.yml file that can be exported and used to install the entire environment.</p><p>There’s a good portability here, mind you, but still cross operating system (OS) installations may break. That’s where <em>docker</em> shines and with a great deal of eco-system components like <em>docker-compose</em> and <em>docker-machine</em>, it is easier than ever to get up and running with a tangible Data Science application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*JWamM6EAaWMQFlcA.jpg" /></figure><p>To start off, we’ll be creating a Dockerfile. As explained above, Dockerfile would take care of setting up the image for you that includes <em>downloading the base images, setting the maintainer name-id, installing the required ubuntu/debian programs, installing language dependencies</em> and many more things.</p><p>We’ll be creating a python3.5 based Docker image that would contain folders data and notebooks with ports 8888 exposed to connect to jupyter notebooks.</p><p>(<em>Note: If any of the python modules can’t be installed via pip, it would be safe to write </em><em>RUN commands to run them safely</em>).</p><p>Create a folder with the tree structure as below:</p><pre>.<br>├── data<br>│   └── readme.md<br>├── Dockerfile<br>├── notebook<br>│   └── readme.md<br>├── README.md<br>└── requirements.txt</pre><p>We have the requirements.txt file as below: (<em>Note: Do </em><em>pip freeze &gt; requirements.txt to generate one for your conda environment</em> )</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/775cbf6d37a4884988b05e35e81b3a5f/href">https://medium.com/media/775cbf6d37a4884988b05e35e81b3a5f/href</a></iframe><p>We’ll start off with getting smaller components for our Dockerfile.</p><h4>base image:</h4><p>To make things simple, the base image that we’ll use is the python:3.5.3-onbuild image. All we need is a requirements.txt file that would have all the python modules to be installed.</p><p>It does the following:</p><ul><li>Install python 3.5.3</li><li>Pull in your source and put it in a conspicuous and runnable place.</li><li>Install your requirements.txt file.</li></ul><p>(<strong>Note: This would create a Debian container, not an Ubuntu one</strong>)</p><pre>FROM python:3.5.3-onbuild</pre><p>Updating the repository sources</p><ul><li>Creating folders for data and notebooks</li></ul><pre>RUN mkdir /data RUN mkdir /notebooks</pre><p>Adding a temporary log folder in /tmp, using:</p><pre>RUN mkdir /tmp/tflearn_logs</pre><p>Make those three folders as VOLUMES so that they can be accessible through the host machine too.</p><ul><li>The main reason we have VOLUMES is that the data and notebooks can be shared separately, not through the images that we create (_Docker best practices states that it isn’t good to share data with Images, it could be done but avoided).</li></ul><pre>VOLUME [&quot;/data&quot;, &quot;/notebooks&quot;, &#39;/tmp/tflearn_logs&#39;]</pre><p>Sometimes we might have installations like curl or cron that aren’t exactly in our base image. We’ll RUN the installations as below:</p><pre>RUN apt-get install cron -yqq \ curl</pre><p>We’ll expose the port:</p><pre># jupyter<br>EXPOSE 8888</pre><p>We’ll have to make an arrangement somehow to start the jupyter notebook as we run our image. For that, we’ll use CMD to run our commands:</p><pre>CMD jupyter notebook --no-browser --ip=0.0.0.0 --allow-root --NotebookApp.token=&#39;demo&#39;</pre><p>Finally, we have our <em>Dockerfile</em> as below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fcfe2be9f4ed0b56628117df27c1344d/href">https://medium.com/media/fcfe2be9f4ed0b56628117df27c1344d/href</a></iframe><p>We need to build the <em>docker image</em> by running the following command:</p><pre>(sudo) docker build -t &lt;image-name&gt; .</pre><p>A simple inspection (below), would give you the statistics for the image generated:</p><pre>user@user:~/docker-demo$ sudo docker images </pre><pre>REPOSITORY   TAG    IMAGE ID       CREATED    SIZE <br>demoimage  latest 9d556dd63886 9 minutes ago 1.37 GB</pre><p>Now the real test comes in, we need to test it by running the image and checking whether our volumes work or not!</p><p>Follow the commands below:</p><pre>(sudo) docker run -d -v ~/docker-demo/notebooks:/notebooks -v ~/docker-demo/data:/data \ -v ~/docker-demo/logs:/tmp/tflearn_logs -p 8888:8888 -i demoimage</pre><p>If you do a sudo docker ps, you would get the following output:</p><pre>CONTAINER ID   IMAGE            COMMAND       CREATED      STATUS PORTS NAMES <br>fadf5e0646d5 demoimage &quot;/bin/sh -c &#39;tenso...&quot; 9 seconds ago <br>Up 8 seconds 0.0.0.0:6006-&gt;6006/tcp, 0.0.0.0:8888-&gt;8888/tcp clever_payne</pre><p>So we are all set to test the docker image:</p><ul><li>Fire up the <a href="http://localhost:8888/">http://localhost:8888</a> to get the jupyter notebook.</li></ul><p>The container and host folders would be synced and persisted, no need to worry about losing the notebook,data and logs.</p><p>The last step would be saving this image to DockerHub as a public image or in a private repository to use it later.</p><pre># Login to DockerHub (_Note: You need to create your DockerHub account first_) <br>(sudo) docker login</pre><p>Once login is succeeded, we need to tag the image and then push the image:</p><pre>(sudo) docker tag &lt;image-name&gt; &lt;login-id&gt;/&lt;image-name&gt; <br>(sudo) docker push &lt;login-id&gt;/&lt;image-name&gt;</pre><p>e.g.</p><ul><li>(sudo) docker tag demoimage pratos/demoimage</li><li>(sudo) docker push pratos/demoimage</li></ul><p>So here it is, your very own <em>docker image</em>. My colleague would be happy to see this, like him even you could do more with <em>docker images</em>. Do try this out.</p><p>We would be looking at a few more things that <em>Docker</em> offers in the next post. To all the <em>DevOps</em> folks do provide feedback to this post and share it with all of us.</p><p><strong>Sources:</strong></p><ol><li><a href="https://www.digitalocean.com/community/tutorials/docker-explained-using-dockerfiles-to-automate-building-of-images">For more indepth and simple understanding on how to write a Dockerfile</a></li></ol><p>2. <a href="https://docs.docker.com/">Docker Documentation</a></p><p>Feel free to share!</p><p><em>Originally published at </em><a href="https://pratos.github.io/docker-for-data-science-part-1.html"><em>pratos.github.io</em></a><em>.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f5f8b%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2F8f5f8b&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href">https://medium.com/media/c43026df6fee7cdb1aab8aaf916125ea/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/1*bQlRSzFHJEmF4Q7PyrLgng.gif" /></figure><figure><a href="https://medium.com/becoming-human/artificial-intelligence-communities-c305f28e674c"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*2f7OqE2AJK1KSrhkmD9ZMw.png" /></a></figure><figure><a href="https://upscri.be/8f5f8b"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*v-PpfkSWHbvlWWamSVHHWg.png" /></a></figure><figure><a href="https://medium.com/becoming-human/write-for-us-48270209de63"><img alt="" src="https://cdn-images-1.medium.com/max/255/1*Wt2auqISiEAOZxJ-I7brDQ.png" /></a></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd41e5ef1d80" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/docker-for-data-science-part-1-dd41e5ef1d80">Simplified Docker-ing for Data Science — Part 1</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Starting to do LISP with Docker Images]]></title>
            <link>https://medium.com/@prato_s/lisp-hmm-40954cf3485a?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/40954cf3485a</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[lisp]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Fri, 21 Apr 2017 18:52:04 GMT</pubDate>
            <atom:updated>2017-04-21T19:00:05.757Z</atom:updated>
            <content:encoded><![CDATA[<p>Randomly browsing through the vast treasure trove of blogs, there seems to come a time when you get fixated on something really special. I happened to come across this <a href="http://www.paulgraham.com/avg.html">very old article by Paul Graham</a> that makes you go — <em>“Hmmm, okay”</em>. Mind you the article that titlliated me talks about the time when The Internet was becoming a sensational thing.</p><p>Came across a meme about Lisp, would I definitely think about this? Maybe no.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*BLzHP_3NKAPvyOxW.jpg" /></figure><p>[Source: Pinterest]</p><p>Digging up old archives and doing Ctrl+Right Click on the related links is my favourite pastime. I kept on munging through the very dusty archives of the web, articles that said <em>LISP was ahead of it’s time</em> and how <em>AI Winter set the rising sun of LISP prematurely to the horizon</em>. Reading things and actually experiencing them are two different beasts. After a short burst of reading about LISP, I kind of made up my mind to get some LISP action.</p><p>One of the oldest programming languages, LISP was the first one to be used for prototyping AI algorithms that were production ready (<a href="http://www.gigamonkeys.com/book/introduction-why-lisp.html">Read this blog post</a>). <em>The name LISP derives from “LISt Processor”. Linked lists are one of Lisp’s major data structures, and Lisp source code is made of lists. Thus, Lisp programs can manipulate source code as a data structure, giving rise to the macro systems that allow programmers to create new syntax or new domain-specific languages embedded in Lisp.</em></p><p>Your very own author is not a ninja hacker, but has done so much Ubuntu setups that he can claim to be a <em>hacker *ahem*</em>. Tried to do a quick installation of SBCL variant of Common LISP through <a href="https://common-lisp.net/project/common-lisp-beginner/sbclinstall_doc.html">this link</a>. Follow the step and you’ll be able to login and write a few commands in your terminal to test the waters.</p><p>With the Jupyter notebooks spoiling us, the act of writing commands in a shell makes me queasy. I went out to search for jupyter kernels that would make my life and documenting LISP easy. I found three good github links:</p><ul><li><a href="https://github.com/robert-dodier/maxima-jupyter">robert-dodier/maxima-jupyter</a></li><li><a href="https://github.com/fredokun/cl-jupyter">fredokun/cl-jupyter</a></li><li><a href="https://github.com/Inaimathi/cl-notebook">Inaimathi/cl-notebook</a></li></ul><p><a href="https://github.com/robert-dodier/maxima-jupyter">robert-dodier/maxima-jupyter</a> and <a href="https://github.com/fredokun/cl-jupyter">fredokun/cl-jupyter</a> provide a way to integrate LISP Kernel in a Jupyter notebook, which makes sense for me (and other users who love Jupyter). <a href="https://github.com/Inaimathi/cl-notebook">Inaimathi/cl-notebook</a>, on the other hand, is a different beast altogether and entirely made on LISP backend, with Javascript frontend. It made me go, <em>“LISP? Hmmm…interesting!”</em></p><p>A live demo <a href="https://vimeo.com/97623064">here</a></p><p><strong>Important note: I failed miserably at getting <em>cl-jupyter</em> working on my laptop hence would update this once I can.</strong></p><p>Living in a world of instant gratification, <em>Docker </em>lets you download docker images and run them to run code without a single installation command. Like with Tensorflow and other obscurely difficult installations, there’s always a kind soul who uploads Docker images for such hassles. The docker image for <em>cl-jupyter</em> was just a Google search away.</p><p>You can find the docker image at <a href="https://hub.docker.com/r/sergiobuj/cl-jupyter/">this link</a>. To get going with Docker, here’s a <a href="https://docs.docker.com/engine/installation/">link</a> for basic Docker installation.</p><p>To get the docker image from DockerHub and run the jupyter notebook having LISP Kernel, follow the steps below:</p><pre>$ docker pull sergiobuj/cl-jupyter $ docker run -d -p 8888:8888 -v /notebooks:/notebooks sergiobuj/cl-jupyter</pre><p>We need to create a repository to map it to /notebooks in the container so that the notebooks that we create during the docker sessions are persisted. To know more about data persistence, <a href="https://www.digitalocean.com/community/tutorials/how-to-work-with-docker-data-volumes-on-ubuntu-14-04">read this</a></p><p>Go to the location: <a href="http://localhost:8888/">http://localhost:8888</a> to find the familiar Jupyter interface. Go on and start programming. For reference, below is the screenshot for my browser:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*b9tLbz03B_rWsFsB.png" /></figure><p>Yet to start off with LISP, but curiosity about it has peaked certainly. Would certainly learn it slowly and be back with an application on it!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40954cf3485a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tensorflow Serving by creating and using Docker images]]></title>
            <link>https://becominghuman.ai/tensorflow-serving-by-creating-and-using-docker-images-336ca4de8671?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/336ca4de8671</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tfserving]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Thu, 06 Apr 2017 21:09:59 GMT</pubDate>
            <atom:updated>2017-04-13T18:02:09.831Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*eR9limL1G3vRGh8RPBeYgA.png" /><figcaption>Tensorflow Docker Images</figcaption></figure><p>Deep Learning (DL) and for a good amount, Machine Learning (ML) suffers from the lack of a proper workflow that makes things simple for the research to directly translate into production. There’s a dearth of easy ways to do it, many cloud providers (Microsoft Azure ML Studio, Google Cloud Platform) and independent servcice providers (BigML) have made doing ML production a bit easy.</p><p>Still there’s a lot of stress to get an ML solution to production. Like how Software Engineering has matured with an eco-system of easing the developer’s load to get things into production, ML has still a way to go. About DL, it still more nascent that traditional ML.</p><p>TensorFlow serving debuted in 2015 and has been garnering more attention within the Deep Learning community. Google has been subtly aggressive about TensorFlow, and that the adoption rate is pretty strong. Statistics says it all:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/0*T4okg-HCVQ5k3WAO.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*C0RTnuvqYXq3aETE.png" /></figure><p>The sheer monstrosity in the volume is evident in the above images. TensorFlow isn’t just the media hype, it gives in so many tools that makes life easy for a Deep Learning researcher. When doing experiments — <a href="https://www.tensorflow.org/get_started/summaries_and_tensorboard"><strong>TensorBoard</strong></a>, it’s integration with other open source projects (namely <strong>Docker, Hadoop, Kubernetes</strong>) and other new things (like <strong>XLA Compiler</strong>) that makes working on TensorFlow exciting. Another important framework that TensorFlow provides us is <a href="https://tensorflow.github.io/serving/"><strong>serving</strong></a>.</p><p>In this part we’ll try to get TensorFlow Serving model server up and running.</p><h4>What is TensorFlow Serving?</h4><blockquote><em>TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out-of-the-box integration with TensorFlow models, but can be easily extended to serve other types of models and data.</em></blockquote><p>At the start of this post, we lamented the dearth of Machine Learning workflows that would make developer and researcher’s lives easier. TensorFlow Serving (tfserving) is a good step forward to do that. It sets up a continuous delivery pipeline, with lesser headaches that include serving a new model instantly, trying to get the requests to old model completed and then discard it. Below is how the tfserving home page describes it:</p><p>tfserving makes it easy for retrained model to deployed to a production environment. It also goes a bit further to make it available to the outside world via a <a href="http://www.grpc.io/">gRPC server</a> making it discoverable. We’ll be making use of tfserving to run Inceptionv3 model.</p><p>This post would deal with creating a custom Docker image (if you aren’t familiar with Docker do read up on that, plus I’ll be writing a few beginner posts on Docker) and pushing it to DockerHub or other managed container services. <em>Why making it as a docker image and pushing it to some repo? You’ll get to know.</em></p><h4>Experience with getting tfserving working</h4><p>In the wintery October 2016, I was tasked with getting tfserving running for our internal research project. Being the naive person I was, tried a lot of things but couldn’t get it working. With a sleuth of projects coming in, tfserving went to the backburner until a few weeks back when shit got real.</p><p>To give you a rough estimate of how much time I took to get tfserving working : 4 months (neglect) + 4 weeks (iterative failures). I’m an Arsenal fan so, 4 is in my horoscope.</p><p>After a lot of reading (not much out there on the web), trail&amp;error success did come in the form of a Docker image.</p><h4>Pre-requisites</h4><p>You could do it on your laptop, provided you have an excellent threshold for patience and absolutely smashing internet connection. I would advise you to use cloud VMs to execute the instructions in this blog post.</p><p>I used Google Cloud’s Compute Engine to create a 30Gb (Standard Disk), Debian GNU/Linux8 OS VM with 2CPUs. This costs me around $49.75. We need a 30Gb disk to commit the images to the disk, had to struggle with this since I was using 10Gb disk. Also, another reason was free $300 Credits to use and no extra to pay.</p><p>You can use DigitalOcean/AWS/MS Azure, your choice or even a local system. We’ll start with the actual execution, ssh in your vm and follow along.</p><h4>1. Downloading the serving repository from tensorflow:</h4><ul><li>git clone --recursive <a href="https://github.com/tensorflow/serving">https://github.com/tensorflow/serving</a></li><li>cd serving</li></ul><h4>2. Installing Docker in Debian:</h4><ul><li>sudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ software-properties-common</li><li>curl -fsSL &quot;https://download.docker.com/linux/debian/gpg&quot;</li><li>sudo apt-key add -</li><li>sudo apt-key fingerprint 0EBFCD88</li><li>sudo add-apt-repository &quot;deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable&quot;</li><li>sudo apt-get update</li><li>sudo apt-get install docker-ce</li><li>apt-cache madison docker-ce</li><li>sudo apt-get install docker-ce=17.03.1~ce-0~debian-jessie</li><li>cd serving</li></ul><p>Try out a dry run by checking:</p><p>(<strong>NOTE:</strong> Add docker user to sudo group follow this <a href="http://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo">link</a>)</p><h4>3. Tweaking the Docker file in serving</h4><p>TensorFlow using Docker file is an easy way to get started without much hassle, but there’s still some things that you need to change. Firstly, there’s no way the port 9000 (<em>used for tensorflow model server where it could be accessed</em>) and with the very recent build the Bazel version needs to be upgraded from 0.4.2 to 0.4.5 so that things don’t break.</p><p>Here are two things that you need to do after doing $ cd tensorflow_serving/tools/docker/:</p><ul><li>Change the Dockerfile.devel by adding EXPORT 9000</li><li>Change the ENV BAZEL_VERSION 0.4.2 to ENV BAZEL_VERSION 0.4.5 (<a href="https://github.com/tensorflow/tensorflow/issues/8738">TensorFlow: Error #8738</a>)</li></ul><h4>4. Building the Docker file</h4><ul><li>docker build --pull -t $USER/tfserving -f ./tensorflow_serving/tools/docker/Dockerfile.devel .</li></ul><h4>5. Running the docker image, do the following:</h4><ul><li>docker run --name=tfserving -p 9000:9000 -d -it $USER/tfserving (-d for detached mode)</li></ul><h4>6. Saving the Docker image</h4><p>This new image would be ~1.09Gb and serves as a base Docker image for tfserving. We’ll be using <a href="https://quay.io/">quay.io</a> to save our images. It has a pretty decent plan, with 30 days free trial. Register and go to the tutorials page to know how to login through the CLI.</p><p>Follow along after logging in:</p><ul><li>docker ps</li></ul><p>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d3f896a1dde6 pratos/tfserving &quot;/bin/bash&quot; 4 seconds ago Up 3 seconds 0.0.0.0:9000-&gt;9000/tcp tfserving</p><ul><li>docker commit d3f896a1dde6 quay.io/pratos/tfserving (Commit the images)</li><li>docker push quay.io/pratos/tfserving (Push the repository to quay.io)</li></ul><p>You’ll find your image in the repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Sm0Yg6uWIG3Tpy7a.png" /></figure><h4>7. Compiling serving repo in the Docker Image</h4><p>This is the most important sections of the process and the most time consuming. The original image would swell from ~1.09Gb to ~5.8Gb, patience and a stable internet connection is needed (advisable to keep this process running in screen or tmux so that it is persisted even when your local internet connection drops off).</p><p>Delete the container you ran previously and create a new one without the -d option or simple attach it (you’ll be able to ssh through docker attach &lt;container&gt;)</p><ul><li>docker run --name=tfserving -p 9000:9000 -it quay.io/pratos/tfserving</li></ul><p>You’ll be in the docker image now, run the commands below carefully to make sure there’s no problem later:</p><ul><li>mkdir -p /work/</li><li>cd /work/</li><li>sudo apt-get install git</li><li>git clone --recurse-submodules <a href="https://github.com/tensorflow/serving">https://github.com/tensorflow/serving</a></li><li>cd /work/serving/tensorflow &amp;&amp; ./configure</li></ul><p>The last command would ask you a few popup questions (most of which would be no). Configure that and run the next set of commands below:</p><ul><li>cd /work/serving</li><li>bazel build -c opt tensorflow_serving/... --jobs 1 --curses no --discard_analysis_cache</li></ul><p>One thing to note here is number of jobs that <em>Bazel</em> spawns is 200, which would lead to virtual memory error. Since, we are on a budget VM (not exactly high end) the number of jobs set is --jobs 1. You could set it higher/tune it accordingly if you want to process to finish faster.</p><p>I experienced a lot of errors when I started with tfserving, after reading up documentation and github issues, was able to crack this thing. Go out for a coffee or do something else once you start the compilation, otherwise waiting for <em>Bazel</em> to compile would feel like the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/0*PdgOhFFQGWAYFxch.jpg" /></figure><p>If you don’t get error, compile is successful and you can run the below command to check:</p><ul><li>/work/serving/bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server</li></ul><pre>usage: /work/serving/bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server </pre><pre>Flags: <br>--port=8500 int32	port to listen on <br>--enable_batching=false bool	enable batching <br>--model_name=&quot;default&quot; string	name of model <br>--model_version_policy=&quot;LATEST_VERSION&quot;	string	</pre><pre>The version policy which determines the number of model versions to be served at the same time. The default value is LATEST_VERSION, which will serve only the latest version. </pre><pre>See file_system_storage_path_source.proto for the list of possible VersionPolicy. </pre><pre>--file_system_poll_wait_seconds=1	int32	</pre><pre>interval in seconds between each poll of the file system for new model version </pre><pre>--model_base_path=&quot;&quot; string	path to export (required) <br>--use_saved_model=false bool	If true, use SavedModel in the server; otherwise, use SessionBundle. </pre><pre>It is used by tensorflow serving team to control the rollout of SavedModel and is not expected to be set by users directly.</pre><h4>8. Testing for InceptionV3 model</h4><p>Our next aim is to make sure that Inceptionv3 works and we are able to get inferences once we provide test images through the server.</p><p>Downloading the InceptionNet models, to test inferences:</p><ul><li>curl -O <a href="http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz">http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz</a></li><li>tar xzf inception-v3-2016-03-01.tar.gz</li><li>bazel-bin/tensorflow_serving/example/inception_saved_model --checkpoint_dir=inception-v3 --output_dir=inception-export</li></ul><p><em>(A hefty output, with warnings peppered handful.)</em></p><pre>WARNING:tensorflow:From /work/serving/bazel-bin/tensorflow_serving/example/inception_saved_model.runfiles/inception_model/inception/inception_model.py:150: histogram_summary (from tensorflow.python.ops.logging_ops) is deprecated and will be removed after 2016-11-30. </pre><pre>Instructions for updating: Please switch to tf.summary.histogram. Note that tf.summary.histogram uses the node name instead of the tag. </pre><pre>This means that TensorFlow will automatically de-duplicate summary names based on the scope they are created in. . . . &lt;WARNINGS CONTINUED&gt; </pre><pre>2017-03-29 19:55:53.938435: W external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn&#39;t compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations. </pre><pre>Successfully loaded model from inception-v3/model.ckpt-157585 at step=157585. <br>Exporting trained model to inception-export/1 </pre><pre>WARNING:tensorflow:From /work/serving/bazel- bin/tensorflow_serving/example/inception_saved_model.runfiles/tf_serving/tensorflow_serving/example/inception_saved_model.py:155: initialize_all_tables (from tensorflow.python.ops.data_flow_ops) is deprecated and will be removed after 2017-03-02. </pre><pre>Instructions for updating: Use `tf.tables_initializer` instead. </pre><pre>Successfully exported model to inception-export</pre><p>Starting the gRPC server:</p><ul><li>bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_name=inception --model_base_path=inception-export &amp;&gt; inception_log &amp;</li></ul><p>Save a few images (any random would do, except for humans I guess) using wget/curl</p><p>Let’s get the inferences:</p><ul><li>bazel-bin/tensorflow_serving/example/inception_client --server=localhost:9000 --image=./images/dog1.jpg</li></ul><pre>D0329 20:43:26.487714047 26684 ev_posix.c:101] <br>Using polling engine: poll outputs <br>{ key: &quot;classes&quot; value <br>   { dtype: DT_STRING tensor_shape <br>        { dim { size: 1 } dim { size: 5 } <br>    } </pre><pre>string_val: &quot;beagle&quot; <br>string_val: &quot;English foxhound&quot; <br>string_val: &quot;Walker hound, Walker foxhound&quot; <br>string_val: &quot;bluetick&quot; <br>string_val: &quot;tennis ball&quot; <br>     } <br>} </pre><pre>outputs <br>{ <br>  key: &quot;scores&quot; <br>  value { dtype: DT_FLOAT tensor_shape { <br>             dim { size: 1 } dim { size: 5 } <br>        } <br>  float_val: 8.8880033493 <br>  float_val: 5.99537038803 <br>  float_val: 5.76968288422 <br>  float_val: 3.91565990448 <br>  float_val: 3.65689516068 <br>  } <br>} </pre><pre>E0329 20:43:29.299981796 26684 chttp2_transport.c:1810] close_transport: {&quot;created&quot;:&quot;@1490820209.299938346&quot;,&quot;description&quot;:&quot;FD shutdown&quot;,&quot;file&quot;:&quot;src/core/lib/iomgr/ev_poll_posix.c&quot;,&quot;file_line&quot;:427}</pre><h4>9. Saving the final Docker container</h4><p>Now that we have the server running on <strong>port 9000</strong>, Ctrl+p and Ctrl+q to quit the Docker container.</p><p>Run the commands below to make sure you commit the containers to images</p><ul><li>docker commit d3f896a1dde6 quay.io/pratos/baseinception (Commit the container to images)</li><li>docker push quay.io/pratos/baseinception (Push the repository to quay.io)</li></ul><p>So in this blog we looked at how to create a new docker image that can be now used just by downloading it from anywhere. DockerHub can also be used for the same, but quay.io felt better (mostly coz I was lazy to search and read up on DockerHub).</p><p>That’s all for now. This blog wouldn’t be possible without the awesome resource on Medium (listed below). I’ve just tried to add a few things that I felt were much doable on budget resources and small tweaking of the base Docker.devel file.</p><p>Feel free to use the docker image: <strong>docker pull quay.io/pratos/baseinception</strong></p><p><strong>Sources:</strong></p><ol><li><a href="https://tensorflow.github.io/serving/docker">TensorFlow using Docker</a></li><li><a href="https://tensorflow.github.io/serving/serving_basic">TensorFlow Serving Basics</a></li><li><a href="https://medium.com/osldev-blog/tensorflow-serving-practical-introduction-9ce29ccd63f">OSL Engineering Blog — TensorFlow Serving : Practical Introduction</a></li><li><a href="https://medium.com/zendesk-engineering/how-zendesk-serves-tensorflow-models-in-production-751ee22f0f4b">Zen Desk Engineering — TensorFlow Serving Blog</a></li><li><a href="https://medium.com/@avloss/tensorflow-serving-in-10-minutes-a0d03f00ebeb">TensorFlow Serving in 10 Minutes</a></li></ol><p>Written on March 31st , 2017 by Prathamesh</p><p>Feel free to share and give feedback!</p><p><em>Originally published at </em><a href="https://pratos.github.io/tensorflow-serving-using-docker.html"><em>pratos.github.io</em></a><em>.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fpowered.by.rabbut.com%2Fp%2F9M4T%3Fc%3D0&amp;src=https%3A%2F%2Fpowered.by.rabbut.com%2Fp%2F9M4T%3Fc%3D0&amp;type=text%2Fhtml&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;schema=rabbut" width="500" height="500" frameborder="0" scrolling="no"><a href="https://medium.com/media/a6edaa982bd7d35e5cbcc4b8ed2f9c2b/href">https://medium.com/media/a6edaa982bd7d35e5cbcc4b8ed2f9c2b/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=336ca4de8671" width="1" height="1" alt=""><hr><p><a href="https://becominghuman.ai/tensorflow-serving-by-creating-and-using-docker-images-336ca4de8671">Tensorflow Serving by creating and using Docker images</a> was originally published in <a href="https://becominghuman.ai/">Becoming Human: Artificial Intelligence Magazine</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CircleCI for Continuous Integration]]></title>
            <link>https://medium.com/@prato_s/circleci-for-continuous-integration-43703251d1f3?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/43703251d1f3</guid>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[circleci]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Tue, 24 Jan 2017 00:00:00 GMT</pubDate>
            <atom:updated>2017-02-02T11:12:46.397Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>After 32 agonizing, illiterate builds the author was able to setup CircleCI for testing, the next 4 he failed in the tests.</em></blockquote><p>When I first entered IT, people who were alloted Testing domains were looked down upon. At the time, “Development” was something that was exciting. Being a noob (along with other noobs), my first impressions about testing were rather very negative.</p><p>I was very wrong when my supposedly “great” code backfired in the deliveries. I didn’t learn from that though since there wasn’t a good test driven environment that my predecessors had set (plus my own disregard for my job). Mostly like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/236/0*zf5BhejjOrSsPlHX.jpg" /></figure><h3>Something that is untested is broken.</h3><p>[<a href="http://flask.pocoo.org/docs/0.12/testing/">Sourced from Flask Testing page</a>]</p><p>Currently things aren’t like that though, <strong>a minor slight might cost a lot</strong> (oh that rhymes!). With the hopes that our folks (again a rhyme!) move to test driven environment, had the opportunity to tinker with <a href="https://circleci.com/">CircleCI</a> for our project.</p><p>To start off, I wrote unit tests for our project code and ran that locally. Windows is the worst environment to test your code if that’s being deployed in a Linux Server (common sense! (bad move, wasted a day to realize that Celery doesn’t work)).</p><p>Below is the project folder structure that I zeroed on for tests. I chose Python’s pytest for the great amount of documentation and relative easy for writing tests (but mostly geek validated).</p><pre>user@userserver:~/server_backend$ tree . ├── circle.yml ├── server_backend │   ├── __init__.py │   ├── __pycache__ │   │   ├── __init__.cpython-35.pyc │   │   ├── server.cpython-35.pyc │   │   └── tasks.cpython-35.pyc │   ├── rabbitmq_celery.md │   ├── server.py │   ├── tasks.py │   └── test │   ├── __pycache__ │   │   └── test_server.cpython-35-PYTEST.pyc │   └── test_server.py ├── makefile ├── README.md └── requirements.txt</pre><p>The tests are quite easy to write down if you have a good understanding of what exactly you want to achieve. Since this blog post is not exactly on how to write tests for your code, would be skipping that. I referred the <a href="http://doc.pytest.org/en/latest/">official documentation</a> and a few blog posts:</p><h3>Getting all this to Github</h3><p>The main goal of this exercise was to setup a Continuous Integration (CI) workflow, wherein when I push a code to the github repo it automatically runs a build and all the tests are run on the code. This would be beneficial when I want to make sure that nothing breaks in the production environment before merging all the latest commits.</p><p>There are a lot of CI frameworks/products available — Jenkins, TravisCI and CircleCI to name a few. CircleCI was an obvious choice since it provided <strong>one free container</strong> to setup our testing shop. Jenkins is open-source, but setup and maintainance could have been a big overhead. CircleCI with it’s tight integration with Github was a promising prospect (<a href="https://infinum.co/the-capsized-eight/continuous-integration-on-android-why-we-ditched-jenkins-for-circle-ci">what I read</a>).</p><h3>Initial setup</h3><p>We need to add the CircleCI application to our Github and give access to it initially. For this go to the CircleCI homepage and signup</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nreCluGojCcxHPC0.png" /></figure><p>You would be redirected to the access page and then to the dashboard that would look like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DIeZY_fODda9Ndba.png" /></figure><p>You can start building your project once you reach here. But there’s a caveat, you need to have some tests in your repo inorder to build the project successfully. (Not very sure, since I had the folder structure messed up one time and the tests did not run the other time).</p><p>Since all our code is housed in a private repository and me not being the admin, I had to wait for the administrator to add the project repo to the build and allow collaborators to include it in their own project builds.</p><h3>circle.yml</h3><p>The major struggle is getting the correct circle.yml for a specified project. CircleCI runs the test in a container (much like a Docker container). For majority of the projects, CircleCI automatically creates containers and runs tests. We had a special case where majority of the setup was to be done carefully.</p><p><a href="https://circleci.com/docs/manually/">Manual configuration of circle.yml</a> was the path we chose. For a python based project, CircleCI supports major releases in 2.7x and 3.5x. We went with Python 3.5.2 (3.5.1 wasn’t available, quite weird and the distributions are circle-python3.5.x format). The VM was Ubuntu 14.04.</p><p>The circle.yml file needs to be added to the root repository. We’ll go stepwise to create a new circle.yml file:</p><h4>Step 1: machine</h4><p>In order to set the specific python installation and assign global variables, machine: is the the parent which should have those keys. In our case, we had to specify python version to be used and create $TF_BINARY_URL (for TensorFlow installs). Our machine: would be as below:</p><pre>machine: python: version: 3.5.2 environment: TF_BINARY_URL: <a href="https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp35-cp35m-linux_x86_64.whl">https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0rc0-cp35-cp35m-linux_x86_64.whl</a></pre><p><em>How should it execute?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*duOF9cAJoK2odFE6.png" /></figure><h4>Step 2: dependencies</h4><p>Now after python installation, we would need specific modules that we need. Since we already had generated requirements.txt file from the virtual environment, the pip install -r requirements.txt proved to be handy.</p><p><strong>(NOTE: You might need to edit the </strong><strong>requirements.txt file if you are using Anaconda distribution and creating virtual environments using </strong><strong>conda create. Remove </strong><strong>conda==&lt;version&gt; and </strong><strong>tensorflow==&lt;version&gt; specifically since it creates problems while setting up the CircleCI VM)</strong></p><p>There are a few dependencies that are pre, post and/or override, we just had to use pre.</p><pre>dependencies: pre: - python --version - pip --version - pip install --upgrade pip - pip install -U $TF_BINARY_URL - pip install -r requirements.txt</pre><p><em>How should it execute?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kwIbtF0GvXEmNCw5.png" /></figure><h4>Step 3: database</h4><p>Databases are the most integral part of any end-to-end application and our application uses mysql as a database. In order to replicate the same set of tables and entries (to mock the test inputs), we ended up overriding the circle_test database i.e. default.</p><p>We created a new user that matches the one in the tasks.py when we connect to our database and carefully generate it. Below is the way it should look:</p><pre>database: override: - mysql -u ubuntu -e &quot;CREATE USER &#39;user1&#39;@&#39;localhost&#39; IDENTIFIED BY &#39;password&#39;;&quot; - mysql -u ubuntu -e &quot;GRANT ALL PRIVILEGES ON * . * TO &#39;user1&#39;@&#39;localhost&#39;;&quot; - mysql -u ubuntu -e &quot;FLUSH PRIVILEGES;&quot; - mysql -h localhost -u &quot;user1&quot; --password=&quot;password&quot; -e &quot;create database athenadr;&quot; - mysql -h localhost -u &quot;user1&quot; --password=&quot;password&quot; -e &quot;show databases;&quot; - mysql -h localhost -u &quot;user1&quot; --password=&quot;password&quot; -e &quot;CREATE TABLE user1.users (id int(11) NOT NULL AUTO_INCREMENT,email varchar(100) NOT NULL,name varchar(100) DEFAULT NULL,tokens varchar(100) DEFAULT NULL,createdat timestamp NULL DEFAULT NULL,PRIMARY KEY (id),UNIQUE KEY email_UNIQUE (email),UNIQUE KEY uid_UNIQUE (id),UNIQUE KEY tokens_UNIQUE (tokens)) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8&quot;</pre><p><em>How should it execute?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yDZnV9KOvQEDr1St.png" /></figure><h4>Step 4: test</h4><p>The final step is the one we are most interested in and should be looked at carefully. This is the place where we add the programming language specific test commands. These commands would trigger the testing modules specific to them. For our project we are using pytest, and a simple pytest would search for the tests in the directory structure to execute them. (CircleCI install nose module automatically for testing)</p><p><em>How should it execute?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6gEY0Rvqv5aA10rC.png" /></figure><h3>Success and How?</h3><p>When you pass the tests, the checks get reflected on your PR (Pull Request) status too as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/0*NppnVes8I1PbB72G.png" /></figure><p>My CircleCI dashboard for the curious.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q69REHKWOWI-m-wr.png" /></figure><p>To wrap this up, my test cases were pretty simple as well as the customizations required by my project were very limited. There’s a lot that can be improved on this, lot of it might be sheer stupidity on my part (it works though). Comments and thoughts are welcomed.</p><h4>Sources and Reading:</h4><ul><li>To know more about .yml format visit <a href="http://ess.khhq.net/wiki/YAML_Tutorial">this link</a></li></ul><p><em>Originally published at </em><a href="https://pratos.github.io/2017/01/24/setting-up-circleci-for-python/"><em>pratos.github.io</em></a><em> on January 24, 2017.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43703251d1f3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Last man on Earth]]></title>
            <link>https://medium.com/@prato_s/last-man-on-earth-166a751938c4?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/166a751938c4</guid>
            <category><![CDATA[fiction]]></category>
            <category><![CDATA[short-story]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Fri, 30 Dec 2016 07:50:07 GMT</pubDate>
            <atom:updated>2016-12-30T07:50:07.965Z</atom:updated>
            <content:encoded><![CDATA[<p><em>The last man on Earth sat alone in a room. There was a knock on the door. The knock was low, polite on the oaken door. The faint echo of the knock faded away as he tried to concentrate on the journal in front of his hand. He squinted to read the handwriting in the small candle light. There was a certain damp in the air, his breathing turning heavy every minute. The knock was unnatural; a bead of sweat trickled down his brow. He sipped the whisky off the jade glass. His throat was still dry though, the chill in the air made him shiver. His strong lower jaw clattered against the upper, was it from fear or the air . . . he wasn’t certain.</em></p><p><em>There was another knock — a heavy hand by the sound of it, this time the wall mount of a tiger loosened away, hanging perilously on the hook. The journal he was reading was that of his linguist friend, Joshua.</em></p><p><em>The doom, the Xhaka, the indigenous tribals believed that a fair devil would walk on these lands. His minions following him, little would he know that the wheel of destruction would be rolled on his kind. The Xhaka. . .</em></p><p><em>Everybody had laughed off Joshua, feeling that the meek guy would piss his pants when he read his journal. Eddard had listened to him but he had never paid him much attention.</em></p><p><em>He was right though, every word of it. He’d seen deaths of all his family, his friends . . . plague they called it, but he knew it was the curse. The ship brought news of the bubonic plague, spread across the known world, taking living souls cheap. This though was not any vermin’s work, the dead! — he squirmed in pain clutching his temples. He felt spent, the fear had numbed him. He couldn’t feel it, the ache in his temples was more excruciating to feel that. He was more frustrated, End it you bastard . . . end it! I’m waiting.</em></p><p><em>This was the same room in his villa where he lived with his wife . . . the scent of her still hung in that room. Was she there outside! She came back for me? Thoughts and tears swelled up, with a quavering voice he asked, “Who’s there? Mary? Are you there? Do you remember your Eddard’s voice?”</em></p><p><em>There was a knock again.</em></p><p><em>“Seven Hells! Go away, go away!” he shouted, he flung the jade glass towards the door. The fever was taking toll on his body; his senses seemed to desert him. He tried to read again.</em></p><p><em>. . . the path of destruction would be laid by him, the agents — fair devil’s own minions. The bloodless revenge for his people, one would kill the other for flesh . . . devouring the meat. Abomination, these minions would be.</em></p><p><em>There was again thumping on the door; this time it was three slaps on the door. He didn’t dare to open the door, he was suddenly feeling frightened. Was it death or something else, he couldn’t decide on that. He could sense the anger of whatever entity was beyond the door.</em></p><p><em>The candle light flickered as his heart pounded, he could hear the silence. The dead . . . , he remembered with index finger rolling the forehead.</em></p><p><em>It can’t be, they are dead . . . I’m alive, I’m the last man. No, no, no. This time there were more bangs on the door, the muffled sounds as if they were calling someone. The Xhakaaaa , he whispered under his breath.</em></p><p><em>Sitting on the chair, he cursed . . . but couldn’t hear his own voice. The bangs on the door were more prominent; the heavy oaken door started vibrating. He realized whatever the entity was; it was trying to break in. The door now creaked with the thuds on the door, every passing minute the door was giving in just like his will. His brow was now covered with sweat; he felt his legs becoming water. He could not think of any other thing. Minutes prior he wanted it to end, but now he wanted some more time. The fever made him sickly, his stomach churned and the headache.</em></p><p><em>The room was now becoming darker, the candle’s glow slowly diminishing. The pounding the door was taking made loud voice; he chose to ignore it then. He returned to the journal amidst the chaos</em></p><p><em>The dead would rise again, the revenge bloodless. The wheels of destruction in flow.</em></p><p><em>With a loud thump the door fell on the floor. The sudden wind blew away the candle. There were black figure, men and women alike staring at him with the hollows in their skull. He could sense the goose bumps on his arm, he felt his bowels out of control and sensed the sphincter had released. The figures were still looking at him, none talking. The room now stank of the waste, he could smell it.</em></p><p><em>Rising from his chair weakly Eddard Bolton walked towards the figures, accepting his fate — his death. As he moved forward, there was a glow of light from the other side — a lantern judging its shape. This felt odd, a figure moved towards him from the other direction with the lantern. He was aghast to find that it was Joyce, his maid. She looked the same — the closed set eyes, small mouth and slender frame. But something was wrong; she was seeing past him . . . he felt something strange.</em></p><p><em>He looked back; a sudden spark went through his forehead. The chair . . . How could . . . him! Me! He laughed hard, laughed hard until he broke down crying on the floor. He saw Eddard Bolton on the chair, the jade glass by his side and the book in his hand lay dead.</em></p><p><em>**************************************************************</em></p><p><em>Sir Eddard Bolton, (1580–1621) resident of Jamestown, Virginia. Author and Philosopher. He was an insomniac and was supposed to have split personality disorder which was an unknown entity in those times. Joshua, his alternate personality through whom he wrote about the Powhatan Tribes and their culture. There were suspicions about the death of his wife, Mary Bolton during the winter of 1609. Mary Bolton was rumored to be murdered by Eddard Bolton for his Cannibalistic theory of Pleasure. Since his wife’s death, he was locked in a room by his nephew, Robert Bolton and attended by Joyce, the maid who brought him food and looked after him. The journal written by him was thought to be his confessions about his crime but every work of Sir Bolton was burned by his nephew who owned the sprawling mansion.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=166a751938c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Stand Alone Spark 2.0 on Windows and Mac]]></title>
            <link>https://medium.com/@prato_s/setting-up-spark-2-0-on-windows-and-mac-35264603bb85?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/35264603bb85</guid>
            <category><![CDATA[mac]]></category>
            <category><![CDATA[windows]]></category>
            <category><![CDATA[hadoop]]></category>
            <category><![CDATA[spark]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Sun, 20 Nov 2016 06:07:03 GMT</pubDate>
            <atom:updated>2016-11-20T07:32:46.149Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TCNR4QVevmr8_jg43Q1L_A.jpeg" /></figure><blockquote>As a Windows user who does machine learning (do not judge me) there’s always a struggle to find some or the other things working on your “beloved” system.</blockquote><p>I have a very minimal configuration packed in a light weight Asus case which has survived about a year of abuse, yet still strong to tell the tale. In this edition we would be looking at how to get up and running with Stand Alone Spark 2.0 (and later) on Windows and Mac OS.</p><blockquote>========================================</blockquote><blockquote><strong><em>Windows 10 (Works for Windows 7 too)</em></strong></blockquote><h4>Java</h4><ul><li>Download &amp; Install Java (latest version) through <a href="https://www.java.com/en/download/-">https://www.java.com/en/download/</a></li><li>Add JAVA_HOME Path in System Variables.</li></ul><pre>C:\Users&gt;java -version<br>java version “1.8.0_91”<br>Java(TM) SE Runtime Environment (build 1.8.0_91-b15)<br>Java HotSpot(TM) 64-Bit Server VM (build 25.91-b15, mixed mode)</pre><blockquote>— — — — — — — — — — — — — — — — — — — — — — — — — —</blockquote><h4>Scala</h4><ul><li>Download and Install Scala (latest version) through <a href="http://www.scala-lang.org/download/">http://www.scala-lang.org/download/</a></li><li>Directly add &lt;Scala location&gt;\bin to the Path.</li></ul><pre>C:\Users&gt;scala -version<br>Scala code runner version 2.10.5 — Copyright 2002–2013, LAMP/EPFL</pre><blockquote>— — — — — — — — — — — — — — — — — — — — — — — — — —</blockquote><h4>Spark</h4><ul><li>We’ll be using Spark prebuilt package for Hadoop 2.6 (later can be used too).</li><li>Download Spark through this link:: <a href="https://spark.apache.org/downloads.html">https://spark.apache.org/downloads.html</a></li><li>Extract the files to a folder (it can be your choice of folder)</li><li>Add the folder path to SPARK_HOME (eg: SPARK_HOME = D:\Hadoop\spark-2.0.1-bin-hadoop2.6) to System Variables as “%SPARK_HOME%\bin”.</li><li>We need to download the ‘winutils.exe’ file and place it in a location (Link to download: <a href="https://github.com/steveloughran/winutils,">https://github.com/steveloughran/winutils,</a> clone the entire folder in your local system and unzip).</li><li>Provide the path for HADOOP_HOME = “Location for winutils files” and add to system variables as “%HADOOP_HOME%\bin”.</li></ul><pre>C:\Users&gt;spark-shell<br>Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties<br>Setting default log level to “WARN”.<br>To adjust logging level use sc.setLogLevel(newLevel).<br>16/11/20 12:05:49 WARN SparkContext: Use an existing SparkContext, some configuration may not take effect.<br>Spark context Web UI available at <a href="http://192.168.59.1:4040/">http://192.168.59.1:4040</a><br>Spark context available as ‘sc’ (master = local[*], app id = local-1479623748733).<br>Spark session available as ‘spark’.<br>Welcome to<br> ____ __<br> / __/__ ___ _____/ /__<br> _\ \/ _ \/ _ `/ __/ ‘_/<br> /___/ .__/\_,_/_/ /_/\_\ version 2.0.1<br> /_/</pre><pre>Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_91)<br>Type in expressions to have them evaluated.<br>Type :help for more information.</pre><pre>scala&gt;</pre><ul><li>You can run various shells which includes the ones for Python (pyspark) and R Programming (SparkR) which will be helpful for Data Science related tasks.</li></ul><blockquote>— — — — — — — — — — — — — — — — — — — — — — — — — —</blockquote><h4>Spark API and connecting it with Jupyter Notebooks</h4><ul><li>You can run Python API on Spark by typing “pyspark” in the command prompt.</li><li>You’ll get a message → Using Python version 2.7.11 (default, Feb 16 2016 09:58:36) SparkSession available as ‘spark’. which confirms the pyspark shell is running properly.</li><li>To connect it to the Jupyter Notebooks we need to set a few Paths in our Windows environment:</li></ul><pre>PYSPARK_DRIVER_PYTHON = ipython</pre><pre>PYSPARK_DRIVER_PYTHON_OPTS = notebook</pre><ul><li>Run the below command to fire up the pyspark shell and make the spark context available directly in IPython Notebook.</li></ul><pre>C:\Users\prato_s&gt;pyspark<br> [TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated and will be removed in future versions.<br>[TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook`… continue in 5 sec. Press Ctrl-C to quit now.<br>[I 12:10:43.302 NotebookApp] The port 8888 is already in use, trying another random port.<br>[I 12:10:43.397 NotebookApp] Serving notebooks from local directory: C:\Users\prato_s<br>[I 12:10:43.397 NotebookApp] 0 active kernels<br>[I 12:10:43.398 NotebookApp] The Jupyter Notebook is running at: <a href="http://localhost:8889/">http://localhost:8889/</a><br>[I 12:10:43.398 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).</pre><ul><li>We can always create and import the pyspark module &amp; sqlContext for our needs.</li></ul><blockquote>========================================</blockquote><blockquote><strong><em>MacOS</em></strong></blockquote><p>I had a chance to setup Spark on MacBook Pro along with my colleague, so this is the documentation for that. Better to start installing with Brew, this makes life easy.</p><ul><li>Open the link <em>(</em><a href="http://spark.apache.org/downloads.html"><em>http://spark.apache.org/downloads.html</em></a><em>)</em> and download it in the folder you wish (in our case it was ~/Downloads)</li><li>Go to bash and execute the following the statements:</li></ul><pre>$ tar xvzf spark-2.0.1-bin-hadoop2.7.tgz </pre><pre>#(Unzip the downloaded folder ) <br>mv spark-2.0.1-bin-hadoop2.7 spark2 </pre><pre>#(Change the filename to spark2) <br>cd spark2/sbin </pre><pre>#(Go to the sbin folder. Open the start-master.sh file. This will start Spark) <br>./start-master.sh</pre><pre>cd ../logs <br>ls #(should show you a link) </pre><pre>tail link #(as given above)</pre><ul><li>The last step will show you that spark as started and give you web link to check the Spark web interface. Spark is successfully installed here.</li><li>Unlike Windows, we’ll be using the module “<a href="https://github.com/minrk/findspark">findspark</a>” to connect to our spark instance. Install the module through pip</li></ul><pre>pip install findspark</pre><ul><li>Fire up the jupyter notebook from bash and write the below code to create spark context:</li></ul><pre>import findspark <br>findspark.init(“/Users/user1/Downloads/spark2/”)</pre><pre>import pyspark <br>from pyspark.sql import DataFrameNaFunctions <br>from pyspark.sql.functions import lit <br>from pyspark.ml.feature import StringIndexer #label encoding <br>from pyspark.ml import Pipeline </pre><pre>sc = pyspark.SparkContext(appName=”helloworld”)</pre><p>You should be able to connect! Hope this is a nice brief documentation about how to setup and anyone following this should be able to setup Stand-alone Spark on their systems properly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=35264603bb85" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Relative Life]]></title>
            <link>https://medium.com/@prato_s/relative-life-5393a8637549?source=rss-9f60702e8437------2</link>
            <guid isPermaLink="false">https://medium.com/p/5393a8637549</guid>
            <category><![CDATA[short-fiction]]></category>
            <category><![CDATA[random-tales]]></category>
            <dc:creator><![CDATA[Prathamesh Sarang]]></dc:creator>
            <pubDate>Sun, 22 Mar 2015 00:00:00 GMT</pubDate>
            <atom:updated>2016-12-30T07:43:54.456Z</atom:updated>
            <content:encoded><![CDATA[<p>There are things that you can never understand in the world I’m going to describe, death and life are just as strange as they come. A parallel universe where people experience relativity and time at a very different level. Things which you like to do tend to make you age faster than things which you do not like.</p><p>People opt for careers that lengthen their lives — everybody fears death and an accelerated death is much dreaded. Fathers outlive strong-headed sons who opt to do things that they like — philosophers they are and optimists to the core.</p><p>Things are very different in this world, more pessimistic you are, the more bitter you are about your life, the more you live — a sad tale of life for people who are bitter, yet they live more than they have to.</p><p>Many people tried to find out why the Creator course to give this life. The philosophers tried to find a way to decelerate this process, but ultimately a their attempts taught us one thing — salvation is a quick process of you want to achieve it, a quicker way to get out of this cycle of rebirth.</p><p>An average lifespan of a male here is 13–140 years and a median of ages is the only way you can generalize the age. An average female can outlive a male and generally take up occupations that need bitter personalities — a trait which the female population picked up from their parallel Earth counterparts from the current 21st century. The term in that universe is ‘bitching’ (<em>Reference: Travel to the Happy Earth — A guide to the dictionary terms in real life by Ekito Kapooro</em>).</p><p>The universe is largely inhabited by humans and their pets (includes insects, serpents and other regular fear inducing mammals). The regular domesticated animals like dogs and cats are moved away from the periphery of the habitation. The lives of people are more like the Poe and Murakami books — strange and dark. The Dark Ages of Earth seem to look happy in comparison with the life here. Fear of death and negativity breeds everywhere and though the humans appear as their Earth version humans, there’s a difference in the mind and heart.</p><p>Technology seems to add to the desire to live more and the marvels of this effort seem to be more popular than music or an act of compassion that may decrease your lifespan. Strange ideals to achieve and stranger rules of life, would it change? Or is it bound to remains as it?</p><p><em>Originally published at </em><a href="http://weekenblog.blogspot.com/2015/03/relative-life.html"><em>weekenblog.blogspot.com</em></a><em> on March 22, 2015.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5393a8637549" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>