<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tiago Lucas on Medium]]></title>
        <description><![CDATA[Stories by Tiago Lucas on Medium]]></description>
        <link>https://medium.com/@tiagoslucas?source=rss-837d4bbf4369------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Kx3alZyFVSmOXpEt7QiGdg.png</url>
            <title>Stories by Tiago Lucas on Medium</title>
            <link>https://medium.com/@tiagoslucas?source=rss-837d4bbf4369------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 03:47:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tiagoslucas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Simple and easy virtualization! Meet Proxmox!]]></title>
            <link>https://tiagoslucas.medium.com/simple-and-easy-virtualization-meet-proxmox-4c89d308f4c5?source=rss-837d4bbf4369------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c89d308f4c5</guid>
            <category><![CDATA[virtual-environment]]></category>
            <category><![CDATA[proxmox]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Tiago Lucas]]></dc:creator>
            <pubDate>Thu, 23 May 2019 20:47:57 GMT</pubDate>
            <atom:updated>2022-05-18T10:00:50.833Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re anything like me, it’s easy to let yourself have a handful of Virtual Machines taking up 2 to 64GBs at a time. So when the time came to create a virtual cluster I saw the list growing before my eyes. Thankfully, for those situations, there is -̶V̶a̶n̶i̶s̶h̶- Proxmox!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*krY60LO1NQqnAKot" /></figure><h4>What exactly is Proxmox? And why the strange name?</h4><p>Asked nobody. Proxmox is a Virtual Environment Software, which basically means you create simple virtual machines intended to work with each other inside it. And according to W̶i̶k̶i̶ a trusted source, its name was chosen for being an available domain at the time of its creation. Kinda kills the mood.</p><h4>How can I set it up?</h4><p>If you haven’t guessed it yet, we are going to create a Virtual Machine to host our Virtual Environment. I will leave it up to the reader to choose the software to do so, since I’m not getting paid by neither one 😏<br>But one thing we all need is the Proxmox Virtual Environment ISO, which can be found here:</p><pre><a href="https://www.proxmox.com/en/downloads/category/iso-images-pve">https://www.proxmox.com/en/downloads/category/iso-images-pve</a></pre><p>On Oracle’s VirtualBox, I created a new Virtual Machine and chose a name and a folder for it. The client type should be “Linux” and the version “Debian (64-bit)”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/1*icS-odo36XVg-1mSNTi41A.png" /></figure><p>Give it a good amount of RAM and disk size, since it will be hosting many machines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*bSbJPh9Bq9FVwx3UXohVIw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*oCOjOrJURsYZg5I9UXsYjA.png" /></figure><p>On our VM Settings lets “insert” the ISO we just downloaded on the CD slot (on Storage, for the ones using Oracle’s VM tool). And boot it up!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*jDQBRDQ3_N1mrXDtXLM8CA.png" /></figure><p>We want to go the “Install Proxmox VE” way, agree with the End User Agreement and select our country :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*-46MR3kUVAGrvZZS-tChwA.png" /></figure><p>Now we choose a password, input our email and let’s get to the serious stuff!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*Fr6Glnc35cu6YMlbPw4VJg.png" /></figure><p>Make sure to select the proper Gateway and DNS Server for your network and choosing an IP Address that’s available on the network. Just don’t forget the IP you chose and then simply click the “Install” button.</p><p>When it asks you to reboot itself, allow it. After it’s done, it will prompt you for login. The username is <strong>root</strong> and the password is the one you defined earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*QM4rGclDz4TLwAUYTsotFA.png" /></figure><p>Insert the given address in your browser’s URL box and it should be ready to use!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c89d308f4c5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a RESTful API with Node.JS]]></title>
            <link>https://tiagoslucas.medium.com/creating-a-restful-api-with-node-js-2fe7a240afc1?source=rss-837d4bbf4369------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fe7a240afc1</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[api-development]]></category>
            <category><![CDATA[loopback]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Tiago Lucas]]></dc:creator>
            <pubDate>Tue, 09 Apr 2019 21:42:13 GMT</pubDate>
            <atom:updated>2019-04-09T21:45:57.119Z</atom:updated>
            <content:encoded><![CDATA[<p>Most of the people using APIs nowadays don’t even know what it means and that’s perfectly normal. Their purpose is to simplify how you get data from the internet. You create your own service that serves data in a nice format (mostly JSON, nowadays) to whoever wants to use it. It’s a pretty simple concept and we’re going to show you that is pretty simple to create one also.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/0*wKTa_u6li9oTds69" /></figure><h4>What’s LoopBack?</h4><p>We’re going to use LoopBack to create our API, mostly because it simplifies the whole process. LoopBack is a framework that asks us for the costumizable values (such as names) and does the rest. Keep it simple, you know?</p><h4>Project Creation</h4><p>You should have Node.JS and a bash terminal installed, both of which you can learn how install on my <a href="https://medium.com/@tiago.slucas/create-your-own-server-with-node-js-5dc02dd030f2">last article</a>. The first thing we’re going to do is create an empty folder and navigate into it using our terminal.</p><p>Installing LoopBack is as easy as writing:</p><pre>npm install -g loopback-cli</pre><p>Creating our project can now be done using the following command:</p><pre>lb</pre><p>It may take a little, but it should prompt us to give our project a name. Mine will be <strong>loopback_api</strong>. Using the arrow keys to select, choose version <strong>2.x</strong> and <strong>api-server</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/1*bOh-M67NMtEPBNMzo1a2vA.png" /></figure><p>Next step is to create a model. You could guess, but that’s easy aswell. Just type:</p><pre>lb model</pre><p>It will give you some choices, but we mainly click Enter and go forward. I choose a model name of <strong>loopback_model</strong>, with <strong>no datasource</strong>, a standart <strong>model</strong>, no custom plural form and <strong>common</strong> for access everywhere. Added a property called <strong>nombre</strong> of type <strong>string</strong>, <strong>required </strong>with <strong>no default</strong>. And we’re set!</p><p>What’s left for us to do is just running our server. Just type:</p><pre> node .</pre><p>Open your browser and go to the url:</p><pre><a href="http://localhost:3000/explorer">http://localhost:3000/explorer</a></pre><p>You’ve just built your own API! If this is so simple, what are kids studying for?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ljfvmDmjmNsLyJicGrwGpg.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fe7a240afc1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create your own server with Node.JS]]></title>
            <link>https://tiagoslucas.medium.com/create-your-own-server-with-node-js-5dc02dd030f2?source=rss-837d4bbf4369------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dc02dd030f2</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[servers]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Tiago Lucas]]></dc:creator>
            <pubDate>Tue, 09 Apr 2019 21:06:23 GMT</pubDate>
            <atom:updated>2019-05-22T14:55:21.856Z</atom:updated>
            <content:encoded><![CDATA[<p>Most of the tech-inclined people know Javascript, the browser-interpreted language that is used on almost every website on the Web. You may already know that JS has a lot of frameworks, enabling you to literally program your way into creating a website, but did you know that you can program your server on that same language? Yes, Javascript is full stack now, something we call a <a href="https://en.wikipedia.org/wiki/MEAN_(software_bundle)">MEAN stack</a>. This article will teach you how to create a simple web server using MongoDB, Express and Node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/1*K6SH2YfF4pdS-wj16b09Nw.png" /></figure><h4>Step 1 —Setting up</h4><p>The first step is to install Node on your machine, going to the <a href="https://nodejs.org/en/download/">Node.JS downloading page</a> and choosing the file that suits your operating system. After installation, create a new folder/directory and start a Node project. You can use your preferred bash terminal (<a href="https://docs.microsoft.com/en-us/windows/wsl/install-win10">Windows has it too, you know?</a>) and simply type:</p><pre>npm init</pre><p>It will ask you some questions about your project but in all honesty I normally just click Enter on every one of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y8RdJAzaxY4SFSiIrzoJAg.png" /></figure><p>Furthermore we want to install our dependecies, the Express framework and Body Parser, for which we’ll use our bash again:</p><pre>npm install --save express body-parser mongoose</pre><p>In this project we want to create a Model-View-Controller architecture to prevent from attacks such as SQL Injection. So we create our folders to each group of files by using our bash again:</p><pre>mkdir models views controllers routes</pre><h4>Step 2 — Files</h4><p>Now that we have a fresh Node project with MVC architecture, lets start by creating the main file:</p><pre>touch index.js</pre><p>And opening that same file with your text editor of choice. On that document we are going to write the following:</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dc02dd030f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transforme os seus Raspberry Pi num cluster de forma rápida e indolor]]></title>
            <link>https://tiagoslucas.medium.com/transforme-os-seus-raspberry-pi-num-cluster-de-forma-r%C3%A1pida-e-indolor-f1b239528c3d?source=rss-837d4bbf4369------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1b239528c3d</guid>
            <dc:creator><![CDATA[Tiago Lucas]]></dc:creator>
            <pubDate>Thu, 04 Apr 2019 21:56:04 GMT</pubDate>
            <atom:updated>2019-04-12T09:11:47.320Z</atom:updated>
            <content:encoded><![CDATA[<p>Com o desenvolvimento da tecnologia, cada vez menos temos interação com o que se passa dentro dos nossos computadores. Uma pessoa que frequente os <em>websites</em> e fóruns de tecnologia, certamente já terá ouvido falar em “cluster”. Mas afinal, o que é um “cluster”? Nada como pôr as mãos na massa…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*4jsx5ST4MjWkbRY9" /><figcaption>Photo by <a href="https://unsplash.com/@kiboka">Nadya Spetnitskaya</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Vamos portanto criar um <em>cluster</em> com Kubernetes.</p><blockquote><a href="https://medium.com/@tavaresdavid19/cluster-com-3-placas-raspberry-pi-a9a828035af9?sk=2fbdee57f6fcf97d89f2543a4a1eedee">Porquê o Kubernetes?</a></blockquote><p>Para este projeto, vamos construir um <em>cluster</em> usando 3 Raspberry Pi. Infelizmente, o suporte para Pi 1 foi descontinuado pelo Kubernetes, portanto todas as máquinas terão de ser posteriores à 1ª geração do <em>Single-Board Computer</em>. Recomendamos vivamente uma ligação à rede por cabo, apesar de ser opcional.</p><h3>1. Instalação</h3><p>O primeiro passo é formatar os cartões microSD. Vamos usar o <em>Raspbian Lite</em>, por ser um sistema operativo leve e supervisionado pela própria <em>Raspberry Pi Foundation</em>. Existe uma grande quantidade de ferramentas gratuitas que podem ser usadas para este processo. Usámos o <em>Win32 Disk Manager</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*0f_ENEU6pvApJqtPeFejKg.png" /></figure><p>Ao iniciarmos o nosso Raspberry devemos mudar o <em>hostname, </em>uma vez que <em>hostnames </em>repetidos no mesmo <em>cluster </em>tendem a gerar problemas. Pode ser feito usando a ferramenta de configuração:</p><pre>sudo raspi-config</pre><p>O Kubernetes funciona com o auxílio de contentores Docker, uma plataforma fácil de instalar:</p><pre>curl -sSL get.docker.com | sh &amp;&amp; sudo usermod pi -aG docker<br>newgrp docker</pre><p>Para assegurar o bom funcionamento do <em>cluster</em>, nas versões mais recentes do Kubernetes, é necessário desativar o espaço de <em>swap</em>:</p><pre>sudo dphys-swapfile swapoff &amp;&amp; sudo dphys-swapfile uninstall &amp;&amp; \ sudo update-rc.d dphys-swapfile remove</pre><p>Também é necessário que o processo <em>cgroup </em>inicie com o computador, que pode ser feito acrescentando comandos ao ficheiro de inicialização:</p><pre>sudo nano /boot/cmdline.txt</pre><p><strong>Sem adicionar nenhuma linha</strong>, é necessário adicionar à linha existente:</p><pre>cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory</pre><blockquote>Agora vamos reiniciar o Raspberry Pi.</blockquote><p>Finalmente, vamos instalar o Kubernetes e descarregar os ficheiros de configuração:</p><pre>curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - &amp;&amp; \<br>echo &quot;deb http://apt.kubernetes.io/ kubernetes-xenial main&quot; | sudo tee /etc/apt/sources.list.d/kubernetes.list &amp;&amp; \<br>sudo apt-get update -q &amp;&amp; sudo apt-get install -qy kubeadm<br>sudo kubeadm config images pull -v3</pre><h3>2. Configuração no “Master”</h3><p>Depois das ferramentas instaladas, vamos configurar o nosso nó <em>master</em>. Começando por iniciar um novo cluster Kubernetes:</p><pre>sudo kubeadm init --token-ttl=0 --pod-network-cidr=10.244.0.0/16</pre><p>Este comando pode demorar até 15 minutos para estar finalizado, dando tempo para se repetir o processo de instalação nos <em>workers</em> do <em>cluster</em>. Os argumentos necessários para a configuração dos mesmos só serão apresentados no final desse tempo. Deve ter vagamente o seguinte aspecto:</p><pre>kubeadm join --token 9e700f.7dc97f5e3a45c9e5 192.168.0.27:6443 --discovery-token-ca-cert-hash sha256:95cbb9ee5536aa61ec0239d6edd8598af68758308d0a0425848ae1af28859bea</pre><p>Como indicado aos que prestam atenção, devemos correr o seguinte <em>snippet</em>:</p><pre>mkdir -p $HOME/.kube<br>sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config<br>sudo chown $(id -u):$(id -g) $HOME/.kube/config</pre><p>De seguida, vamos configurar o <em>flannel</em>, a ferramenta de auxílio de rede que usamos neste caso:</p><pre>kubectl apply -f <a href="https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml">https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml</a><br>sudo sysctl net.bridge.bridge-nf-call-iptables=1</pre><h3>3. Configuração nos Workers</h3><p>Assumindo que o <em>worker</em> já está configurado e o comando de <em>join</em> já foi dado pelo <em>kubeadm </em>no <em>master</em>, devemos usar o mesmo em cada <em>worker</em>. De seguida, para utilizar o <em>flannel</em>, usamos o seguinte:</p><pre>sudo sysctl net.bridge.bridge-nf-call-iptables=1</pre><p>Devemos agora ter o nosso cluster a funcionar. Isso pode ser verificado dando o seguinte comando no <em>master</em>:</p><pre>kubectl get nodes</pre><p>Se todos os nós aparecerem na lista, o seu <em>cluster</em> está montado.</p><blockquote><a href="https://medium.com/@pedromelopppp/testar-o-cluster-dos-raspberry-pi-cluster-criado-com-o-kubernetes-2fa175dba1d9">Pode agora testar o seu cluster!</a></blockquote><p>Referência: <a href="https://blog.alexellis.io/serverless-kubernetes-on-raspberry-pi/">https://blog.alexellis.io/serverless-kubernetes-on-raspberry-pi/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1b239528c3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Set up Hadoop Cluster on VirtualBox machines running CentOS 7]]></title>
            <link>https://tiagoslucas.medium.com/set-up-hadoop-cluster-on-virtualbox-machines-running-centos-7-df9121b1eab0?source=rss-837d4bbf4369------2</link>
            <guid isPermaLink="false">https://medium.com/p/df9121b1eab0</guid>
            <category><![CDATA[centos7]]></category>
            <category><![CDATA[hadoop]]></category>
            <category><![CDATA[cluster]]></category>
            <category><![CDATA[apache]]></category>
            <dc:creator><![CDATA[Tiago Lucas]]></dc:creator>
            <pubDate>Wed, 20 Mar 2019 22:56:29 GMT</pubDate>
            <atom:updated>2022-05-18T10:07:17.909Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SdvWc-pRqmZHDWXgYLOivg.png" /></figure><blockquote>In this tutorial I’m going to show you how to create your own Hadoop cluster with VirtualBox, using three CentOS virtual machines.</blockquote><h4>1. Setting up</h4><p>First, you’re gonna have to create a virtual machine on VirtualBox and install CentOS 7 on it. On our new virtual machine, the first thing we want to do is setup the network. For that, we’re going to use the command:</p><pre>nmtui</pre><p>It should open something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*sXFRI9Acj0dAYgw9oC2dOg.png" /><figcaption>Network Manager start screen</figcaption></figure><p>Let’s select “Edit a connection” and select one. It should have a name like “enp0s3”, at least that’s what I have on my VM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*aZObG9YbShaXorxGyJiN6A.png" /></figure><p>You should set the IPv4 and IPv6 configurations to Automatic and select “Automatically connect” (select with Space). Then just press “OK”. If it went all fine you should now be able to ping “1.1.1.1”, Cloudflare’s DNS server. When you’re satisfied just press Ctrl + C to exit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*Qcsj3mSAZcAmhiqPNcpPxw.png" /></figure><p>Now we’re going to install Java. For that you simply have to type:</p><pre>yum install java-1.8.0-openjdk-devel -y</pre><h4>2. Installing Hadoop</h4><p>It’s recommended that we create a “super user” to use hadoop, so we’ll do that by using the following commands:</p><pre>adduser hadoop<br>passwd hadoop<br>usermod -aG wheel hadoop</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*MZ0kOiC6mqTu8SfI8gLM3g.png" /><figcaption>Creating the Super User “hadoop”</figcaption></figure><p>Next on we’re going to create a secure shell key and add it to “authorized keys” to enable an ssh connection to our machine. That can be done in the hadoop user by accessing it:</p><pre>su - hadoop</pre><p>And then just using the following commands:</p><pre>ssh-keygen -t rsa<br>cat ~/.ssh/id_rsa.pub &gt;&gt; ~/.ssh/authorized_keys<br>chmod 0600 ~/.ssh/authorized_keys</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*acS0KDEbed5FVrtPzrNc6A.png" /><figcaption>Generating and authorizing an ssh key</figcaption></figure><p>Now, on downloading the latest version of hadoop, we need to get the url from the website: <a href="https://hadoop.apache.org/releases.html">https://hadoop.apache.org/releases.html</a>. Once there, you select the latest binary and the following page will present you the URL. In my case it looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cg69cQOow-9JB8p_QBI0Ew.png" /></figure><p>Using the <em>curl </em>and <em>tar</em> commands we download hadoop from that URL and extract it:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*DUr668C-KUC3d-HmdYeepg.png" /></figure><p>I’m going to rename my hadoop folder to simply “hadoop” (it’s a matter of issue):</p><pre>mv hadoop-3.1.2 hadoop</pre><p>Moving on we need to set some constants in our .bashrc file. That can be done by opening it with <em>vi</em>:</p><pre>vi ~/.bashrc</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*_tItWFpxwnCX_rvKquM07Q.png" /><figcaption>Vi text editor</figcaption></figure><p>Inside <em>vi</em>, you just press “a” (the append shortcut) on your keyboard and add the following to the bottom of the file:</p><pre><strong>export</strong> HADOOP_HOME=<strong>/</strong>home<strong>/</strong>hadoop<strong>/</strong>hadoop<br><strong>export</strong> HADOOP_INSTALL=$HADOOP_HOME<br><strong>export</strong> HADOOP_MAPRED_HOME=$HADOOP_HOME<br><strong>export</strong> HADOOP_COMMON_HOME=$HADOOP_HOME<br><strong>export</strong> HADOOP_HDFS_HOME=$HADOOP_HOME<br><strong>export</strong> YARN_HOME=$HADOOP_HOME<br><strong>export</strong> HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME<strong>/</strong>lib<strong>/</strong>native<br><strong>export</strong> PATH=$PATH:$HADOOP_HOME<strong>/</strong>sbin:$HADOOP_HOME<strong>/</strong>bin</pre><blockquote>Now, note that in my case the folder is just called “hadoop”, but that may not be the case if you didn’t change it!</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*dRmuyOgvutNR-lNS5sjrSA.png" /></figure><p>Then just press Esc and write “:wq” (for write and quit). That should save the file. However, to change the current environment, we have to use the command:</p><pre>source ~/.bashrc</pre><p>To make sure everything is set right we can use:</p><pre>ls $HADOOP_HOME</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/797/1*eLzatP494olkXGt22fFiWA.png" /></figure><p>In case you didn’t get the same results as I did you should go back and review your ~/.bashrc file.</p><h4>3. Setting up Hadoop</h4><p>Now that we have hadoop on our machine we should configure it. This is where we use the Java that we installed in the beginning of this tutorial.</p><p>First things first, we should navigate to the folder where all the configuration files should be by using:</p><pre>cd $HADOOP_HOME/etc/hadoop</pre><p>The first file to edit is called “ hadoop-env.sh” and we’re going to use <em>vi</em> again:</p><pre>vi hadoop-env.sh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*4Tyakyfl_3Mrb-eTDnvjAQ.png" /></figure><p>We only need to add the line:</p><pre>export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk</pre><p>It should be irrevelant where to put it, as long as it’s a new line. Then just write and quit <em>vi</em>. Finally, open and write for the following files:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*RP_pdHT4LiY_fGdFghqG1g.png" /><figcaption>core-site.xml</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*bQ24fh7I-BT328dkq2OF2A.png" /><figcaption>hdfs-site.xml</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*imibbW3qTRzAlgRjej7MwA.png" /><figcaption>mapred-site.xml</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*Aj5lWFMK5J80Qsf0TbZVXQ.png" /><figcaption>yarn-site.xml</figcaption></figure><p>With that done we return to our default directory with the command “cd”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*ymuX6elvFKG1NnRcfOsHRA.png" /><figcaption>“pwd” means “print working directory”</figcaption></figure><h4>4. Setting Up Multi-node</h4><p>To use a Multi-node setup we must define the nodes in our <em>hosts</em> file, which can be achieved using <em>vi</em> once more:</p><pre>vi /etc/hosts</pre><p>All we need to add are the nodes’ names and respective IP addresses. In my case I’m using three nodes, one<em> </em>“master” and two “workers”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/797/1*8Y0eVoZNZAXbOm4ObH5eEw.png" /><figcaption>hosts file</figcaption></figure><p>Now we’re going to create those machines. To begin we have to clone the “master” machine twice:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/1*H-i3f647yEyrCCvkoLmMxA.png" /><figcaption>Using the Right-Click, we select the “Clone” option</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/749/1*0fPvXtuGSdgiRLg4QCUWUg.png" /><figcaption>After choosing the name and folder to the new machine, VirtualBox will start cloning</figcaption></figure><p>You should now have three virtual machines. For them to be able to communicate with each other, we’re going to open VirtualBox’s Network Settings:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/1*qPB5aGuC3qMwwI5Xl8fDpw.png" /><figcaption>Using Right-Click again, we select “Settings”</figcaption></figure><p>On “Network”, we select “Adapter 2&quot;, enable and select Host-Only Adapter. Let’s set “Promiscuous Mode” to “Allow All” and be careful that the MAC Adresses are different from each other:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*Jz7kT22mME9fDWy3Jv5Ckw.png" /></figure><p>Each Virtual Machine should have a different hostname from the ones we defined in the <em>hosts </em>file. Log in with the hadoop account directly for this part.</p><p>The rest is simpler than what we’ve already done with <em>vi</em>:</p><pre>vi /etc/hostname</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*FkwHGnvPTL3drmyQpyc3jw.png" /></figure><p>To further configure that, we have to use <em>nmtui </em>like at the start of this tutorial:</p><pre>nmtui</pre><p>In the new connection, “Wired Connection 1” in my case, we’ll simply set IPv4 to “Manual”, press “Show” and input the IP Address for the respective machine followed by “/24”. This is on my “master” VM:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*G-0u8FbPK9JF2NSASppnzA.png" /></figure><p>Simply press “OK” and reboot. Rinse and repeat for each VM. We should now have a network between our machines:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4NlhytdbJTWclCTuO68K0w.png" /></figure><p>Now let’s inform Hadoop about the existing nodes. On our “master” machine we should use the “hadoop” user to edit the <em>workers </em>file:</p><pre>vi $HADOOP_HOME/etc/hadoop/workers</pre><p>In this file we need to write the machines that will do the work:</p><pre>master<br>slave1<br>slave2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*O6yUMgICJyrJNmVE7qV9jg.png" /></figure><p>Finally, with all the machines on, we start our cluster by using the following commands on the “master” machine:</p><pre>$HADOOP_HOME/etc/hadoop/start-dfs.sh<br>$HADOOP_HOME/etc/hadoop/start-yarn.sh</pre><p>But don’t take my word for granted, open one of your “worker” machines and use the <em>jps</em> command:</p><pre>jps</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/313/1*CP4SzVa3g5HLtQoBIkxrkw.png" /></figure><p>If you see “NodeManager” and “DataNode” there, your cluster is up and running 🙂</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df9121b1eab0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>