<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Soufiane Bouchaara on Medium]]></title>
        <description><![CDATA[Stories by Soufiane Bouchaara on Medium]]></description>
        <link>https://medium.com/@soufianebouchaa?source=rss-61cca0053586------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*aLnA2fDldFrtcbyWDs_evw.jpeg</url>
            <title>Stories by Soufiane Bouchaara on Medium</title>
            <link>https://medium.com/@soufianebouchaa?source=rss-61cca0053586------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:13:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@soufianebouchaa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Freeswitch CI/CD with GitHub Actions, DroneCI [PART 2]]]></title>
            <link>https://soufianebouchaa.medium.com/freeswitch-ci-cd-with-github-actions-droneci-part-2-6ca86017d59c?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ca86017d59c</guid>
            <category><![CDATA[freeswitch-cicd]]></category>
            <category><![CDATA[freeswitch-build]]></category>
            <category><![CDATA[freeswitch]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[freeswitch-github-actions]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 31 Dec 2022 19:18:36 GMT</pubDate>
            <atom:updated>2022-12-31T19:18:36.479Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rb4h_gy5QrVgDOZw" /><figcaption>Photo by <a href="https://unsplash.com/@jramos10?utm_source=medium&amp;utm_medium=referral">Josue Isai Ramos Figueroa</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This series of articles is for Freeswitch developers, to help them achieve a better &amp; modern CI/CD.</p><p><a href="https://medium.com/@soufianebouchaa/freeswitch-ci-cd-on-ubuntu-22-04-debian-11-part-1-75fb1be97316">In the previous article (Part 1)</a> we installed Freeswitch across Ubuntu 22.04 LTS &amp; Debian 11 bullseye, with better handling of dependencies, also we were able to generate FreeSWITCH .deb files, and store them into an s3 bucket for future deployments.</p><p>A reminder to the goal of this series of articles is to cover :</p><ul><li><a href="https://medium.com/@soufianebouchaa/freeswitch-ci-cd-on-ubuntu-22-04-debian-11-part-1-75fb1be97316"><strong>PART 1:</strong> Install Freeswitch across Ubuntu 22.04 LTS &amp; Debian 11 bullseye, Better handling of dependencies, generate FreeSWITCH.deb files, and push them into an s3 bucket for future deployments.</a></li><li><strong>PART 2: </strong>Continuous Integration (CI) using GitHub Actions or DroneCI.</li><li><strong>PART 3: </strong>Adding Freeswitch sounds &amp; custom ringtones to the FreeSWITCH.deb installer.</li><li><strong>PART 4: </strong>Continuous Deployment (CD) for FreeSWITCH deployment with Ansible.</li></ul><p>So, let’s create a file called <strong>freeswitch-build.sh</strong> in the root folder of our repository.</p><p>This file should be called from any CI tool (Github Actions or Drone CI for our case), and will contains all the commands we saw in the part 1 of this series, in order to generate a .deb file but this time it will be automated after every push command :</p><pre>#!/bin/sh<br>set -e<br><br>apt update -y <br>apt upgrade -y <br>DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata<br>apt install curl wget git subversion build-essential autoconf automake libtool libncurses5 libncurses5-dev make libjpeg-dev libtool libtool-bin libsqlite3-dev libpcre3-dev libspeexdsp-dev libldns-dev libedit-dev yasm liblua5.2-dev libopus-dev cmake libtiff-dev  libcodec2-dev libcodec2-dev portaudio19-dev libmagickcore-dev libmp3lame-dev libmpg123-dev libshout3-dev libvlc-dev libpq-dev libmariadb-dev libldap2-dev erlang librabbitmq-dev libsmpp34-dev libyaml-dev libmongoc-dev libopencv-dev  libmemcached-dev libavformat-dev  libh2o-dev libsoundtouch-dev libhiredis-dev libopus-dev  autoconf automake devscripts gawk gettext libcurl4-openssl-dev libdb-dev libedit-dev libgdbm-dev libldns-dev libncurses5-dev libopus-dev libopus-ocaml libpcre3-dev libperl-dev libpq-dev libspeex-dev libspeexdsp-dev libssl-dev libtiff5-dev libtool libtool-bin libvorbis0a libogg0 libsqlite3-dev libogg-dev libvorbis-dev portaudio19-dev libshout3-dev libmpg123-dev libmp3lame-dev yasm libbsd-dev flite flite1-dev libflite1 liblua5.2-0 liblua5.2-dev lua5.2 luarocks libsndfile-dev -y<br><br>#gcc g++ <br>apt install gcc-10 g++-10 cpp-10 -y <br>ln -s /usr/bin/gcc-10 /usr/bin/gcc <br>ln -s /usr/bin/g++-10 /usr/bin/g++<br><br>#openssl<br>mkdir -p /opt/openssl <br>wget https://www.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate -P /opt/openssl/<br>tar -xzvf /opt/openssl/openssl-1.1.1q.tar.gz -C /opt/openssl/ <br>cd /opt/openssl/openssl-1.1.1q<br>./config <br>make install<br>cp /usr/local/bin/openssl /usr/bin/openssl<br><br><br>#spandsp<br>cd /opt<br>git clone https://github.com/freeswitch/spandsp /opt/spandsp<br>cd /opt/spandsp<br>./bootstrap.sh<br>./configure<br>make<br>make install <br><br><br><br>#sofia-sip<br>cd /opt<br>git clone https://github.com/freeswitch/sofia-sip.git /opt/sofia-sip<br>cd /opt/sofia-sip<br>./bootstrap.sh<br>./configure<br>make<br>make install <br><br><br>cd /opt<br>git clone https://github.com/freeswitch/freeswitch.git -b v1.10.8 /opt/freeswitch<br>cd /opt/freeswitch<br>./bootstrap.sh -j<br>CFLAGS=-Wno-error ./configure --prefix=/usr/local/freeswitch --enable-core-pgsql-support --enable-zrtp<br><br>#here you can customize the modules to load in Freeswitch by editing this file : /opt/freeswitch/modules.conf<br>make<br><br>#generate deb file<br><br>mkdir -p /opt/freeswitch/freesiwtch-v1.10.8/DEBIAN<br>cat &gt; /opt/freeswitch/freesiwtch-v1.10.8/DEBIAN/control &lt;&lt; EOL<br>Package: FreeSWITCH <br>Version: v1.10.8<br>Architecture: all<br>Maintainer: Soufiane Bouchaara &lt;contact@soufianebouchaara.com&gt;<br>Description: FreeSWITCH<br>EOL<br><br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freesiwtch/lib/<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freeswitch/lib<br>cp /opt/freesiwtch/freeswitch /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp /opt/freesiwtch/fs_cli /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp -r /opt/freesiwtch/.libs /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp -r /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/.libs /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freesiwtch/lib<br>dpkg-deb --build --root-owner-group /opt/freesiwtch/freesiwtch-v1.10.8<br><br># freesiwtch-v1.10.8.deb is ready to be pushed to S3 Bucket</pre><h3><strong>GitHub Actions</strong></h3><blockquote>GitHub Actions is <strong>a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline</strong>.</blockquote><p>You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production.</p><p>For that, let’s create the folder workflows inside .github folder.</p><p>Then let’s create the file <em>build.yaml</em> to define the build steps :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*utCkfPBRt3Z5uyD2mG6u6g.png" /></figure><p>The file build.yaml should be as follow :</p><pre>name: Build Freeswitch<br>on:<br>  push:<br>  pull_request:<br><br>jobs:<br>  build:<br>    runs-on: ubuntu-22.04<br><br>    steps:<br>    - uses: actions/checkout@v2<br>    - run: bash freeswitch-build.sh<br>        <br>    - name: Configure AWS credentials<br>      uses: s3-actions/s3cmd@v0<br>      with:<br>        provider: &quot;aws&quot;<br>        access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        secret_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        region: us-east-1<br>        <br>    - name: Copy deb file to S3 Bucket<br>      run: |<br>        s3cmd sync freesiwtch-v1.10.8.deb s3://xxxxxx/freeswitch/freesiwtch-v1.10.8.deb</pre><p>Secrets should be stored inside your repository -&gt; Settings -&gt; Actions -&gt; secrets :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KtsVgjdfSntFOEODUEj0sA.png" /></figure><p>then create the following 2 secrets :</p><ul><li>AWS_ACCESS_KEY_ID</li><li>AWS_SECRET_ACCESS_KEY</li></ul><h3>Drone CI</h3><blockquote>Drone CI is <strong>an open-source CI server acquired by Harness</strong>. It is used to build and test software, either on a developer’s local machine or in a continuous integration environment like any public cloud.</blockquote><p>With Drone, you can automate your software delivery pipeline from code commit to production</p><p>Using Drone CI, the YAML configuration file should be as follow:</p><pre>---<br>kind: pipeline<br>type: kubernetes<br>name: freeswitch<br>service_account_name: drone-runner-aws-privileges<br><br>steps:<br>  - name: Compile Freeswitch (AMD Architecture)<br>    image: ubuntu:22.04<br>    resources:<br>      limits:<br>        cpu: 2000<br>        memory: 4096MiB    <br>    commands:<br>      - /bin/bash freeswitch-build.sh<br>      </pre><p>You can also build a deb file for ARM Architecture:</p><pre>- name: Compile Freeswitch (ARM Architecture)<br>    image: arm64v8/ubuntu:22.04<br>    resources:<br>      limits:<br>        cpu: 2000<br>        memory: 4096MiB    <br>    commands:<br>      - /bin/bash freeswitch.sh</pre><p>to Upload your .deb file to S3 bucket :</p><pre>upload:<br>    image: plugins/s3<br>    source: foo/bar.txt<br>    target: /<br>    bucket: my-test-bucket-1234<br>    secrets: [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY]</pre><p>Thank you for your time, reading this article, the next articles will cover the following topics :</p><ul><li><strong>PART 3: </strong>Adding Freeswitch sounds &amp; custom ringtones to the FreeSWITCH.deb installer.</li><li><strong>PART 4: </strong>Continuous Deployment (CD) for FreeSWITCH deployment with Ansible.</li></ul><p>If you face any issues, please feel free to contact me at contact[at]soufianebouchaara.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ca86017d59c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Freeswitch CI/CD on Ubuntu 22.04 & Debian 11 [PART 1]]]></title>
            <link>https://soufianebouchaa.medium.com/freeswitch-ci-cd-on-ubuntu-22-04-debian-11-part-1-75fb1be97316?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/75fb1be97316</guid>
            <category><![CDATA[cicd]]></category>
            <category><![CDATA[ubuntu-22-04-lts]]></category>
            <category><![CDATA[telecom]]></category>
            <category><![CDATA[freeswitch]]></category>
            <category><![CDATA[freeswitch-software]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 24 Dec 2022 08:48:34 GMT</pubDate>
            <atom:updated>2022-12-31T19:19:34.439Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t2p2Ni7EJIVHL8Cz" /><figcaption>Photo by <a href="https://unsplash.com/@quinoal?utm_source=medium&amp;utm_medium=referral">Quino Al</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This series of articles is for Freeswitch developers who fork the official repositories and add their own features, requiring custom compilations every time. Also for those who need FreeSWITCH on the current Linux distributions.</p><p>Note: to achieve this we have to downgrade some essential components such as OpenSSL, and GCC in order to accomplish the compilation of Freeswitch.</p><p>The following tutorials will enable you to install Freeswitch on AMD and ARM architectures, as I do on daily basis.</p><p>Our goal in this series of articles is to cover in :</p><ul><li><strong>PART 1:</strong> Install Freeswitch across Ubuntu 22.04 LTS &amp; Debian 11 bullseye, Better handling of dependencies, generate FreeSWITCH.deb files, and push them into an s3 bucket for future deployments.</li><li><a href="https://soufianebouchaa.medium.com/freeswitch-ci-cd-with-github-actions-droneci-part-2-6ca86017d59c"><strong>PART 2: </strong>Continuous Integration (CI) using GitHub Actions or DroneCI.</a></li><li><strong>PART 3: </strong>Adding Freeswitch sounds &amp; custom ringtones to the FreeSWITCH.deb installer.</li><li><strong>PART 4: </strong>Continuous Deployment (CD) for FreeSWITCH deployment with Ansible.</li></ul><p><strong>Step 1 :</strong> (Skip if you already forked the Freeswitch repository)<strong> :</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K61qvH0ZtCxy7QBvPwIcNw.png" /><figcaption>Forking Freeswitch repository</figcaption></figure><p><strong>Step 2: Compiling Freeswitch on a Ubuntu 22.04 LTS VM</strong></p><p>Let’s update the packages of the VM first :</p><pre>apt update -y <br>apt upgrade -y <br>DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata</pre><p>Installing Freeswitch dependencies (ubuntu 22.04)</p><pre>apt install curl wget git subversion build-essential autoconf automake libtool libncurses5 libncurses5-dev make libjpeg-dev libtool libtool-bin libsqlite3-dev libpcre3-dev libspeexdsp-dev libldns-dev libedit-dev yasm liblua5.2-dev libopus-dev cmake libtiff-dev  libcodec2-dev libcodec2-dev portaudio19-dev libmagickcore-dev libmp3lame-dev libmpg123-dev libshout3-dev libvlc-dev libpq-dev libmariadb-dev libldap2-dev erlang librabbitmq-dev libsmpp34-dev libyaml-dev libmongoc-dev libopencv-dev  libmemcached-dev libavformat-dev  libh2o-dev libsoundtouch-dev libhiredis-dev libopus-dev  autoconf automake devscripts gawk gettext libcurl4-openssl-dev libdb-dev libedit-dev libgdbm-dev libldns-dev libncurses5-dev libopus-dev libopus-ocaml libpcre3-dev libperl-dev libpq-dev libspeex-dev libspeexdsp-dev libssl-dev libtiff5-dev libtool libtool-bin libvorbis0a libogg0 libsqlite3-dev libogg-dev libvorbis-dev portaudio19-dev libshout3-dev libmpg123-dev libmp3lame-dev yasm libbsd-dev flite flite1-dev libflite1 liblua5.2-0 liblua5.2-dev lua5.2 luarocks libsndfile-dev -y</pre><p>We have to downgrade GCC to version 10 instead of the latest version to have a working compilation :</p><pre>apt install gcc-10 g++-10 cpp-10 -y <br>ln -s /usr/bin/gcc-10 /usr/bin/gcc <br>ln -s /usr/bin/g++-10 /usr/bin/g++</pre><p>We have also to downgrade OpenSSL to version1 :(, I have read in their GitHub Issues that they are working on a new release to support Opnessl v3</p><pre>mkdir -p /opt/openssl <br>wget https://www.openssl.org/source/openssl-1.1.1q.tar.gz --no-check-certificate -P /opt/openssl/<br>tar -xzvf /opt/openssl/openssl-1.1.1q.tar.gz -C /opt/openssl/ <br>cd /opt/openssl/openssl-1.1.1q<br>./config <br>make install<br>cp /usr/local/bin/openssl /usr/bin/openssl</pre><p>Let’s install Spandsp &amp; Sofia-sip which are the two very essential dependencies to compile Freeswitch.</p><p><strong>SpanSDP (</strong>low-level signal processing library that modulates and demodulates signals commonly used in telephony, such as the “noise” generated by a fax modem or DTMF touchpad<strong>)</strong></p><pre>cd /opt<br>git clone https://github.com/freeswitch/spandsp /opt/spandsp<br>cd /opt/spandsp<br>./bootstrap.sh<br>./configure<br>make<br>make install </pre><p><strong>Sofia-sip (</strong>SIP User-Agent library, compliant with IETF RFC3261<strong>)</strong></p><pre>cd /opt<br>git clone https://github.com/freeswitch/sofia-sip.git /opt/sofia-sip<br>cd /opt/sofia-sip<br>./bootstrap.sh<br>./configure<br>make<br>make install </pre><h4><strong>Freeswitch compilation :</strong></h4><pre>cd /opt</pre><p>Clone from the official repository or from your forked repository, you can also replace the master to checkout the branch you wish to use.</p><p>we will checkout to the latest version released <a href="https://github.com/signalwire/freeswitch/releases/tag/v1.10.8">v1.10.8</a> :</p><pre>git clone https://github.com/freeswitch/freeswitch.git -b v1.10.8 /opt/freeswitch</pre><pre>cd /opt/freeswitch<br>./bootstrap.sh -j</pre><p>Then we run the configure command, here I choose to use ZRTP, and PostgreSQL, you can edit as you wish :</p><pre>CFLAGS=-Wno-error ./configure --prefix=/usr/local/freeswitch --enable-core-pgsql-support --enable-zrtp</pre><p>Now you can edit the following files to choose the modules to use in FreeSWITCH( I highly recommend putting only the modules needed for better performance/troubleshooting)</p><p>Here’s my list of modules :</p><p>/opt/freeswitch/modules.conf :</p><pre>applications/mod_abstraction<br>applications/mod_bert<br>applications/mod_blacklist<br>applications/mod_callcenter<br>applications/mod_cidlookup<br>applications/mod_cluechoo<br>applications/mod_commands<br>applications/mod_conference<br>applications/mod_curl<br>applications/mod_db<br>applications/mod_directory<br>applications/mod_distributor<br>applications/mod_dptools<br>applications/mod_easyroute<br>applications/mod_enum<br>applications/mod_esf<br>applications/mod_esl<br>applications/mod_expr<br>applications/mod_fifo<br>applications/mod_fsk<br>applications/mod_fsv<br>applications/mod_hash<br>applications/mod_httapi<br>applications/mod_http_cache<br>applications/mod_lcr<br>applications/mod_sms<br>applications/mod_snom<br>applications/mod_sonar<br>applications/mod_spandsp<br>applications/mod_spy<br>applications/mod_stress<br>applications/mod_valet_parking<br>applications/mod_voicemail<br>applications/mod_voicemail_ivr<br>codecs/mod_amr<br>codecs/mod_b64<br>codecs/mod_g723_1<br>codecs/mod_h26x<br>codecs/mod_opus<br>codecs/mod_vpx<br>dialplans/mod_dialplan_asterisk<br>dialplans/mod_dialplan_xml<br>endpoints/mod_loopback<br>endpoints/mod_portaudio<br>endpoints/mod_rtc<br>endpoints/mod_rtmp<br>endpoints/mod_skinny<br>endpoints/mod_sofia<br>endpoints/mod_verto<br>event_handlers/mod_cdr_csv<br>event_handlers/mod_cdr_sqlite<br>event_handlers/mod_event_socket<br>event_handlers/mod_format_cdr<br>event_handlers/mod_json_cdr<br>formats/mod_local_stream<br>formats/mod_native_file<br>formats/mod_png<br>formats/mod_portaudio_stream<br>formats/mod_shout<br>formats/mod_sndfile<br>formats/mod_tone_stream<br>loggers/mod_console<br>loggers/mod_logfile<br>loggers/mod_syslog<br>say/mod_say_en<br>xml_int/mod_xml_cdr<br>xml_int/mod_xml_curl<br>xml_int/mod_xml_rpc<br>xml_int/mod_xml_scgi<br>databases/mod_pgsql</pre><p>then we run the make command to start our compilation, it should take from 5 to 15mins depends on the capabilities of the VM :</p><pre>make</pre><p>Once our make command is finished successfully, we are not going to run <em>make install </em>since we want to generate a .deb file that is installable elsewhere.</p><h4><strong>Deb Creation</strong></h4><p>The current latest (when writing this article) is version released is <a href="https://github.com/signalwire/freeswitch/releases/tag/v1.10.8">v1.10.8</a></p><pre>mkdir -p /opt/freeswitch/freesiwtch-v1.10.8/DEBIAN<br><br>cat &gt; /opt/freeswitch/freesiwtch-v1.10.8/DEBIAN/control &lt;&lt; EOL<br>Package: FreeSWITCH <br>Version: v1.10.8<br>Architecture: all<br>Maintainer: Soufiane Bouchaara &lt;contact@soufianebouchaara.com&gt;<br>Description: FreeSWITCH<br>EOL<br><br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freesiwtch/lib/<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin<br>mkdir -p /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freeswitch/lib<br>cp /opt/freesiwtch/freeswitch /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp /opt/freesiwtch/fs_cli /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp -r /opt/freesiwtch/.libs /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/<br>cp -r /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/bin/.libs /opt/freesiwtch/freesiwtch-v1.10.8/usr/local/freesiwtch/lib<br>dpkg-deb --build --root-owner-group /opt/freesiwtch/freesiwtch-v1.10.8</pre><p>The deb file must be created, you can check it as follow :</p><pre>ls freesiwtch-v1.10.8.deb</pre><p>You can copy it to your S3 bucket for further deployments :</p><pre>aws s3 cp freesiwtch-v1.10.8.deb s3://&lt;your bucket name&gt;</pre><p>The next articles will cover the following topics :</p><ul><li><a href="https://soufianebouchaa.medium.com/freeswitch-ci-cd-with-github-actions-droneci-part-2-6ca86017d59c"><strong>PART 2: </strong>Continuous Integration (CI) using GitHub Actions or DroneCI.</a></li><li><strong>PART 3: </strong>Adding Freeswitch sounds &amp; custom ringtones to the FreeSWITCH.deb installer.</li><li><strong>PART 4: </strong>Continuous Deployment (CD) for FreeSWITCH deployment with Ansible.</li></ul><p>If you face any issues, please feel free to contact me at contact[at]soufianebouchaara.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75fb1be97316" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What do monitoring metrics tell us?]]></title>
            <link>https://soufianebouchaa.medium.com/what-do-monitoring-metrics-tell-us-81e6b4f18dcf?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/81e6b4f18dcf</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[influxdb]]></category>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[telegraf]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Fri, 18 Mar 2022 13:32:32 GMT</pubDate>
            <atom:updated>2022-03-18T13:32:32.005Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IlkBGrUGuzsbEj0K" /><figcaption>Photo by <a href="https://unsplash.com/@chrisliverani?utm_source=medium&amp;utm_medium=referral">Chris Liverani</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Monitoring is a wide topic that provides the real-time status of the health of targeted applications, services, infrastructure ..etc.</p><p>Real-time streaming of these metrics and visualizing them into graphs are the most crucial parts of the monitoring service.</p><h3>Introduction</h3><p>In this article, we will cover what metrics can tell us about what’s happening inside our production environment. We will assume that we have the following stack installed across all of our infra :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ijPmttMBLpdEsSMX.png" /></figure><p><strong>Telegraf: </strong>Used to collect metrics and push them to an output(influxDB), <a href="https://www.influxdata.com/time-series-platform/telegraf/">Get Telegraf Now</a></p><p><strong>InfluxDB:</strong> Time-Serie Database optimized to store metrics in time , <a href="https://docs.influxdata.com/influxdb/v2.1/install/">Get InfluxDB Now</a></p><p><strong>Grafana:</strong> Used to visualize data from different sources, and create amazing real-time charts (Influxdb in our case), <a href="https://grafana.com/docs/grafana/latest/installation/">Get Grafana Now</a></p><h3>What Metrics Should I collect?</h3><p>All metrics matters for investigations, analysis, troubleshooting, preventive monitoring, but here are a list of the must-collected metrics:</p><p><strong>Server Resources:</strong> RAM, CPU, Load, AVG Load, Disk, Disk IO, Temperature</p><p>For that we should configure our telegraf as follow :</p><pre># Read metrics about cpu usage<br>[[inputs.cpu]]<br>## Whether to report per-cpu stats or not<br>percpu = true<br>## Whether to report total system cpu stats or not<br>totalcpu = true<br>## If true, collect raw CPU time metrics.<br>collect_cpu_time = true<br>## If true, compute and report the sum of all non-idle CPU states.<br>report_active = true<br></pre><pre># Read metrics about disk usage by mount point<br>[[inputs.disk]]<br>## By default stats will be gathered for all mount points.<br>## Set mount_points will restrict the stats to only the specified mount points.<br># mount_points = [&quot;/&quot;]</pre><pre>## Ignore mount points by filesystem type.<br>ignore_fs = [&quot;tmpfs&quot;, &quot;devtmpfs&quot;, &quot;devfs&quot;, &quot;overlay&quot;, &quot;aufs&quot;, &quot;squashfs&quot;]<br></pre><pre># Read metrics about disk IO by device<br>[[inputs.diskio]]<br></pre><pre># Get kernel statistics from /proc/stat<br>[[inputs.kernel]]<br># no configuration</pre><pre># Read metrics about memory usage<br>[[inputs.mem]]<br># no configuration</pre><pre># Get the number of processes and group them by status<br>[[inputs.processes]]<br># no configuration</pre><pre># Read metrics about system load &amp; uptime<br>[[inputs.system]]<br># no configuration</pre><pre># # Get kernel statistics from /proc/vmstat<br>[[inputs.kernel_vmstat]]<br>#   # no configuration</pre><pre># # Provides Linux sysctl fs metrics<br>[[inputs.linux_sysctl_fs]]<br>#   # no configuration</pre><p>These metrics can be nicely represented as follows :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lVpNFMzLZCmg_oqb.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NXMfiqVg3b4hOoXm.png" /></figure><p><strong>Network: </strong>Cards, uplinks</p><p>For that we should configure our telegraf as follow :</p><pre># # Read metrics about network interface usage<br>[[inputs.net]]<br>#   ## By default, telegraf gathers stats from any up interface (excluding loopback)<br>#   ## Setting interfaces will tell it to gather these explicit interfaces,<br>#   ## regardless of status.<br>#   ##<br>#   # interfaces = [&quot;eth0&quot;]<br>#   ##</pre><pre># Read TCP metrics such as established, time wait and sockets counts.<br>[[inputs.netstat]]<br># no configuration</pre><p>These metrics can be represented as follow :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*g9O3iDbhpyn15GPU.png" /></figure><p>From the screenshot, we will easily understand that users are using this server to download file because we have some spikes on the eth:OUT graph.</p><p><strong>Applications: </strong>Systemd units, Files, or directories:</p><pre>[[inputs.systemd_units]]<br>## Set timeout for systemctl execution<br>timeout = &quot;25s&quot;</pre><p>The procstat plugin can be used to monitor the system resource usage of one or more processes. The procstat_lookup metric displays the query information, specifically the number of PIDs returned on a search.</p><p>In the following example, we will collect metrics for Systemd Units and</p><pre># Monitor process cpu and memory usage for <br>[[inputs.procstat]]<br>  ## PID file to monitor process Nginx<br>  pid_file = &quot;/var/run/nginx.pid&quot;</pre><pre>[[inputs.procstat]]<br>  ## Unit Name to monitor process SSHD<br>   systemd_unit = &quot;sshd.service&quot;<br>   <br>[[inputs.procstat]]<br>  ## Unit Name to monitor process IPFS<br>   systemd_unit = &quot;ipfs.service&quot;</pre><p>In Grafana we can see the data as Follow :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kSAd2uWOEeG84r8s.png" /></figure><p>This will give us a very detailed window to understand what is causing a shortage of resources, and how our software is behaving (for example we can detect a memory leak and understand where it comes from). For more documentation of <a href="https://github.com/influxdata/telegraf/tree/master/plugins/inputs/procstat">procstate plugin</a>.</p><p>Database metrics are also important, let’s take Postgres for example</p><pre>[[inputs.postgresql]]<br>  ## specify address via a url matching:<br>  address = &quot;host=localhost user=postgres sslmode=disable&quot;</pre><pre>  # databases = [&quot;app_production&quot;, &quot;testing&quot;]</pre><pre>  ## with pool_mode set to transaction.<br>  prepared_statements = true</pre><p>The metrics can be fetched on Grafana as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JjqreGhFNWvVSF8A.png" /></figure><p>We can see the Queries Per Second, And Postgres Rows by Operations(Fetched, Returned, Inserted, Updated, Deleted).</p><p>So this will help us understand what’s really going on backstage, allowing us to act quickly in order to save the health of the server and promote our business benefit.</p><p>More documentation for <a href="https://github.com/influxdata/telegraf/tree/master/plugins/inputs/postgresql">postgresql here</a>,</p><h3>Uses Cases</h3><p><strong><em>CPU spike:</em> </strong><br>A CPU spike is <strong>a sudden increase in processor utilization</strong>, which can cause temporary or permanent damage to the CPU and motherboard. Spikes can be caused by the simultaneous running of applications that use a large number of resources and RAM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/471/0*PelP5p0AcsIOujqA.png" /></figure><p><strong><em>Disk IO Spike :</em> </strong><br>Disk I/O spikes are extreme changes in disk demands that can cause huge impacts on the server.<br>Disk I/O Spike can be due to Sudden high usage of Databases Read or Right Which can be related to an intended cause(promotion for example) or to an unintended cause(Attack Attempt or Scrape )</p><p><strong><em>Memory leaks:</em></strong><br>A memory leak occurs when the software or the app create a memory in the heap and does not free it during the time, the cumulation of the reserved RAM increases, the following screenshot shows a real memory leak:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/0*oYvJFGwcu_qWTJjV.png" /></figure><p>The consequence of memory leak is that it brings down the performance of the server by shrinking down the available memory which can cause an Out-Of-Memory and lead the server to be unreachable in a few minutes or even seconds.</p><h3>Conclusion</h3><p>In this article, we discovered what’s the most used stack to collect metrics from servers, and what are the main metrics that we should collect in order to observe, monitor, and prevent our server from any future problems.</p><p>The next part of this article will cover more uses cases and explain them in-depth.</p><p><em>If you have a problem or questions </em><a href="http://contact@soufianebouchaara.com/"><em>Feel free to contact me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81e6b4f18dcf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[SLA and SLO fundamentals and how to calculate SLA]]></title>
            <link>https://soufianebouchaa.medium.com/sla-and-slo-fundamentals-and-how-to-calculate-sla-7f3e75831f79?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f3e75831f79</guid>
            <category><![CDATA[sla]]></category>
            <category><![CDATA[sre]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[slo]]></category>
            <category><![CDATA[nginx]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 05 Mar 2022 15:44:09 GMT</pubDate>
            <atom:updated>2022-03-05T15:44:09.950Z</atom:updated>
            <content:encoded><![CDATA[<p>SLA aka <strong>S</strong>ervice-<strong>L</strong>evel <strong>A</strong>greement is an agreement you make with your clients/users, which is a measured metric that can be time-based or aggregate-based.</p><h3>SLA time-based</h3><p>We can calculate the <strong>tolerable duration of downtime</strong> to reach a given number of nines of availability, using the following formula:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/0*H-gt2tK2TBlPR1Oj" /></figure><p>For example, a web application with an availability of <strong>99.95%</strong> can be down for up to <strong>4.38 hours max in a year</strong>.</p><p>The following table explains the maximum duration of tolerated downtime per year/month/week/day/hour.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IgBPZs2tjrOayO9m.png" /></figure><h3>SLA Aggregate-based</h3><p>Let’s imagine a Backend with an API, that <strong>serves 250M requests per day,</strong> and an SLA Aggregate-based of <strong>99.99% </strong>which cannot exceed more than <strong>25k errors per day</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/0*scoeGqmYf_Zkaqf3" /></figure><p><strong>Note that an error is counted if it’s an internal server error HTTP 5XX</strong></p><h3>What is SLO?</h3><p>SLO aka <strong>S</strong>ervice-<strong>L</strong>evel <strong>O</strong>bjective is an SLA agreement that defines the expectations and goals that the company should achieve during a defined period of time.</p><p><strong>Example:</strong></p><p>Imagine a company with a current <strong>Uptime-Based SLA </strong>is<strong> 95% </strong>which means they have a tolerated <strong>maximum of 1.5days of downtime per month.</strong> (Which is very bad)</p><p>We will define our next <strong>objectives</strong> so the SLA should meet<strong> 99%.</strong></p><p>For that, we have to take several actions, here are some examples:</p><ul><li>Review the hardware infrastructure</li><li>Add preventive monitoring</li><li>Find the root causes of the downtimes</li><li>Review the network configurations</li><li>Review for any single point of failure</li></ul><h3>Let’s Calculate our SLA from Webserver Logs</h3><p>In this example, we are going to collect Nginx Logs and ship them to Elasticsearch in order to visualize them in Kibana to create an SLA Dashboard</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JL6gEglzHGzpb3Hl.png" /></figure><p><strong>Why vector as logs collector?</strong> because it’s a lightweight, ultra-fast tool for building observability pipelines, where we will collect, transform, and route <em>the Nginx</em> logs <em>to Elasticsearch.</em></p><p><strong>Step 1: </strong>Configure your Nginx to provide more detailed logs<br>Edit <strong>/etc/nginx/nginx.confg</strong> in <strong>http</strong> and block add the following :</p><pre>log_format apm &#39;&quot;$time_local&quot; client=$remote_addr &#39;<br>               &#39;method=$request_method request=&quot;$request&quot; &#39;<br>               &#39;request_length=$request_length &#39;<br>               &#39;status=$status bytes_sent=$bytes_sent &#39;<br>               &#39;body_bytes_sent=$body_bytes_sent &#39;<br>               &#39;referer=$http_referer &#39;<br>               &#39;user_agent=&quot;$http_user_agent&quot; &#39;<br>               &#39;upstream_addr=$upstream_addr &#39;<br>               &#39;upstream_status=$upstream_status &#39;<br>               &#39;request_time=$request_time &#39;<br>               &#39;upstream_response_time=$upstream_response_time &#39;<br>               &#39;upstream_connect_time=$upstream_connect_time &#39;<br>               &#39;upstream_header_time=$upstream_header_time&#39;;</pre><p>On your Nginx server definition edit the log configuration as follow:</p><pre>server {<br>		....<br>        access_log /var/log/nginx/access.log apm;<br>        error_log /var/log/nginx/error.log;<br>		....<br>}</pre><p><strong>Step 2: </strong>Install <a href="https://vector.dev/">Vector</a> to collect, parse and ship your Nginx Logs.</p><pre>curl --proto &#39;=https&#39; --tlsv1.2 -sSf https://sh.vector.dev | bash</pre><p><a href="https://vector.dev/docs/setup/installation/">Click for more installation documentation</a></p><p>Using VRL (Vector Remap Langage) to parse the Nginx Logs:</p><pre>[sources.nginx_source]<br>type = &quot;file&quot;<br>ignore_older_secs = 600<br>include = [ &quot;/var/log/nginx/error.log&quot; ]<br>read_from = &quot;beginning&quot;<br>max_line_bytes = 102_400<br>max_read_bytes = 2_048</pre><pre>[transforms.modify_logs]<br>type = &quot;remap&quot;<br>inputs = [&quot;nginx_source&quot;]</pre><pre>#parse each line with VRL regex<br>source = &quot;&quot;&quot;<br>. = parse_regex!(.message, r&#39;^\&quot;(?P&lt;timestamp&gt;.*)\&quot; client=(?P&lt;client&gt;.*) method=(?P&lt;method&gt;.*) request=\&quot;(?P&lt;method_type&gt;.*) (?P&lt;request_path&gt;.*) (?P&lt;http_version&gt;.*)\&quot; request_length=(?P&lt;request_length&gt;.*) status=(?P&lt;status&gt;.*) bytes_sent=(?P&lt;bytes_sent&gt;.*) body_bytes_sent=(?P&lt;body_bytes_sent&gt;.*) referer=(?P&lt;referer&gt;.*) user_agent=\&quot;(?P&lt;user_agent&gt;.*)\&quot; upstream_addr=(?P&lt;upstream_addr&gt;.*) upstream_status=(?P&lt;upstream_status&gt;.*) request_time=(?P&lt;request_time&gt;.*) upstream_response_time=(?P&lt;upstream_response_time&gt;.*) upstream_connect_time=(?P&lt;upstream_connect_time&gt;.*) upstream_header_time=(?P&lt;upstream_header_time&gt;.*)$&#39;)</pre><pre>#covnert metrics to proper types<br>.timestamp = to_timestamp!(.timestamp)<br>.request_length = to_int!(.request_length)<br>.status = to_int!(.status)<br>.bytes_sent = to_int!(.bytes_sent)<br>.body_bytes_sent = to_int!(.body_bytes_sent)<br>.upstream_status = to_int!(.upstream_status)<br>.request_time = to_float!(.request_time)<br>.upstream_response_time = to_float!(.upstream_response_time)<br>.upstream_connect_time = to_float!(.upstream_connect_time)<br>.upstream_header_time = to_float!(.upstream_header_time)<br>.host = &quot;YOUR_HOSTNAME_HERE&quot;</pre><pre>&quot;&quot;&quot;</pre><pre>#for debug mode only - output to console<br>#[sinks.debug_sink]<br>#type = &quot;console&quot;<br>#inputs = [&quot;modify_logs&quot;]<br>#target = &quot;stdout&quot;<br>#encoding = &quot;json&quot;</pre><pre>#OUPUT 1 : Elasticsearch<br>[sinks.send_to_elastic]<br>type = &quot;elasticsearch&quot;<br>inputs = [ &quot;modify&quot; ]<br>endpoint = &quot;https://elasticsearch-endpoint.com&quot;<br>index = &quot;websitelogs-%F&quot;<br>mode = &quot;bulk&quot;<br>auth.user=&quot;xxxxxxxxxxxxxxxxxxxxxxx&quot;<br>auth.password=&quot;xxxxxxxxxxxxxxxxxxx&quot;<br>auth.strategy=&quot;basic&quot;</pre><pre>systemctl restart vector.service</pre><p>Check out the logs of the vector systemd unit :</p><pre>journalctl -u vector.service</pre><p>it should look like the following :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SmGIKl6b48wGy6AU.png" /></figure><p>For demo purposes, I have deployed an Elasticsearch instance in the <a href="https://blog.kalvad.com/sla-and-slo-fundamentals/cloud.elastic.co/">elastic cloud</a></p><h3>Define your current SLA before defining your SLO</h3><p>In Kibana, we need to create an Index Pattern to read from Elasticsearch indexes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bwBo4q865d9Qgn89.png" /></figure><p>Note: The timestamp should be the one from logs, not the Time of Ingest(default one by Elasticsearch).</p><p>Our logs are successfully being shipped into Elasticsearch, so the next step is to create a dashboard in Kibana with some graphs in order to calculate our current SLA.</p><p><strong>Let’s discover the logs</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j9QHJ0WMJh9siM1x.png" /></figure><p>Well, it looks like we got around 8300 for the hit last 15 minutes.</p><p>and every hit log is well prepared and looks as follow :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rQgawBPTOd7nGF0I.png" /></figure><p>Let’s create our first graph, the count of Hits:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Rj4D6zwOTlhL3XrI.png" /></figure><h3>Calculate your Aggregate-based</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*x3idIPLGVla_EqzW.png" /></figure><p>So as we agreed on the formula above, we will consider only 5XX as failed requests, and the rest of the status codes are successful (4XX are considered as client behavior).</p><p>so Our SLA Aggregate-based during the last 15mins is <strong>87.04% </strong>and during the last 30 days is as follow (<strong>98.56%</strong>):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6uvQZMPztlhlYV8T.png" /></figure><p>This gap of difference between the last 15 mins and the last 30 days leads us to understand that an incident is going on.</p><p>Our Dashboard will start to look like this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tacX8KBAfdXLub11.png" /></figure><h3>Using Metrics instead of Logs</h3><p>Metrics are so powerful than Logs, you can use them to get real-time dashboards.<br>In this tutorial, we used Nginx logs and Elasticsearch as a document-store database.<br>But for better performance and real-time dashboards, I highly recommend using metrics instead of logs.</p><p>For that, we can use time-series databases such as InfluxDB or Warp10 to store our metrics and use Grafana as a visualization tool.</p><p>On vector to configure the output sink to InfluxDB by adding the following bloc:</p><pre>#OUPUT 2 : InfluxDB Database<br>[sinks.influxdb_output]<br>type = &quot;influxdb_logs&quot;<br>inputs = [ &quot;modify_logs&quot; ]<br>bucket = &quot;vector-bucket&quot;<br>consistency = &quot;any&quot;<br>database = &quot;xxxxxxxxxxx&quot;<br>endpoint = &quot;https://your-endpoint.com&quot;<br>password = &quot;your-password-here&quot;<br>username = &quot;username&quot;<br>batch.max_events=1000<br>batch.timeout_secs=60<br>namespace = &quot;service&quot;</pre><p>then restart your vector service</p><h3>Conclusion</h3><p>In this article, we discovered what is SLA and SLO and how SLA Aggregate-based can be calculated from Nginx Log.<br>I will cover in the next article how to calculate SLA time-based, and how to improve the SLA by finding the root causes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f3e75831f79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IPFS in-depth]]></title>
            <link>https://medium.com/coinmonks/ipfs-in-depth-87cfef633f2b?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/87cfef633f2b</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[nft]]></category>
            <category><![CDATA[ipfs]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 05 Mar 2022 15:38:32 GMT</pubDate>
            <atom:updated>2022-03-11T02:59:30.248Z</atom:updated>
            <content:encoded><![CDATA[<h3>First of all, What is IPFS?</h3><p>The Interplanetary File System (IPFS) is a bundle of subprotocols and a project-driven by <strong>Protocol Labs, </strong>IPFS aims to improve the web’s efficiency and to make the web <strong>more decentralized</strong> and resilient.</p><p>IPFS uses content-based addressing, where content is not addressed via a location but via its content. IPFS stores and addresses data with its deduplication properties, allowing efficient storage of data. It also can be used as a storage service complementing blockchains, enabling different applications on top of IPFS.</p><p><strong>Why is it called Interplanetary File Systems? What was the reason behind its creation?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XSZDqgztMvDheII3" /></figure><p>IPFS Originally was created to allow access to files from anywhere — whether it be Mars or any other planet</p><p>Alright, so communicating from Mars to Earth is quite difficult, and time-taxing, requesting a Wikipedia page could take up to 48 minutes, which is very time-consuming. (<a href="https://blog.kalvad.com/myths-about-ipfs/Source%20https://marspedia.org/Internet">Source</a>)</p><p>So this was the first reason behind creating IPFS in order to bridge the gap between the planets due to its distributed storage system.</p><p>For example, if a colleague had already requested the Wiki page Mars, you could download it from them easily, just as fast as you would on Earth.</p><h3>First, we need to understand a few concepts :</h3><p><strong>1/ Centralized vs Decentralized Networks</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*Kt_tgTjxgHpoQm_G" /></figure><p><strong>2/ Location-Base Addressing</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IU39GZD6FN7SzJ3m.png" /></figure><p><strong>3/ Content-Base Addressing:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jrOG74g__OliMQwY.png" /></figure><p><strong>How does IPFS work? How exactly do you access content on IPFS?</strong></p><p>IPFS is a peer-to-peer network for storing content on a distributed file system.</p><p>A set of computers/servers called <strong>nodes</strong> that store and relay content using a common addressing system.</p><p>IPFS nodes communicate via the Internet using a Peer-to-Peer (P2P) architecture, <strong>preventing one node from becoming a single point of failure.</strong></p><p>libp2p is a modular system of <em>protocols</em>, <em>specifications</em>, and <em>libraries</em> that enable the development of peer-to-peer network applications <a href="https://docs.libp2p.io/introduction/what-is-libp2p/">— libp2p documentation</a>.</p><p>That’s exactly what <strong>four IPFS nodes need to connect to the IPFS network</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cO6V6W6XPsN3fCpn.png" /></figure><p>Any node that we maintain a connection with is <strong>a peer</strong>.</p><ul><li>Any peer node <strong>can ask their peers</strong>, they want a tomato 🍅 content.</li><li>so, If you do have it, <strong>you will give </strong>🍅<strong> to them</strong>.</li><li>Else <strong>you can give some metadata about the network</strong> to help them find another peer who might have it.</li><li>Note that each node chooses the resources they want to keep and store<strong>.</strong></li><li>Example: 🍅 will be available on different peers nodes who<strong> dedicated resources to store it</strong>.</li><li>A 🥔 exists less in the network. so, only a few nodes will provide it.</li></ul><p><strong>Introducing IPFS Gateway</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*okrET8wGaWHhong0.png" /></figure><ul><li>A gateway is an HTTP interface to an IPFS node.</li><li>On our gateway, we will allow any user to retrieve arbitrary content from IPFS Network.</li><li>If a user asks for a<strong> </strong>🍅 content, nodes with<strong> </strong>🍅 will provide. and the same example goes for the 🥔.</li></ul><p><strong>IPFS Objects</strong></p><p>A Link structure has three data fields:</p><ul><li><strong><em>Name</em></strong> — the name of the Link.</li><li><strong><em>Hash</em></strong> — the hash of the linked IPFS object.</li><li><strong><em>Size</em></strong> — the cumulative size of the linked IPFS object</li></ul><p>IPFS objects are normally referred to by their Base58 encoded hash.</p><p>Exemple: IPFS object with hash <strong>QmeKsf9PmpH4LaJYG3fcBqddjCckuEFC8LHs67Wrg59d51</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kRxYLUAFjfoxu0eA.png" /></figure><h3>What is happening?</h3><p>As we can see in the following picture, Our Kalvad photo will be converted into a binary file, then using IPFS technology this binary file will be split into different chunks.</p><p>Each chunk has its own Hash called CID(Content Identifier), grouped by a Base CID.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BHvBpKOpFqJy3Iuu.png" /></figure><h3>Let’s get our hands dirty, Installation steps :</h3><p>All installation instructions are available here: <a href="https://docs.ipfs.io/install/command-line/#official-distributions">https://docs.ipfs.io/install/command-line/#official-distributions</a></p><pre>wget <a href="https://dist.ipfs.io/go-ipfs/v0.10.0/go-ipfs_v0.10.0_linux-amd64.tar.gz">https://dist.ipfs.io/go-ipfs/v0.10.0/go-ipfs_v0.10.0_linux-amd64.tar.gz</a></pre><p>Unzip the file:</p><pre>tar -xvzf go-ipfs_v0.10.0_linux-amd64.tar.gz</pre><p>Move into the go-ipfs folder and run the install script:</p><pre>cd go-ipfs</pre><pre>sudo bash install.sh</pre><p>Test that IPFS has been installed correctly:</p><pre>ipfs --version<br>&gt; ipfs version 0.10.0</pre><h3>Creating our first IPFS object</h3><p>Let’s introduce our folder :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/0*DkXuv-NpuWHpAxYr.png" /></figure><p>Where pizza.jpg is duplicated (inside food folder and in the ipfs-data folder</p><p>to add a folder into the IPFS network we have to run the following command:</p><pre>ipfs add ipfs-data -r</pre><p>The flag -r for recursive add, the result should looks like this :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wLV4ZYuEhZNE--T7.png" /></figure><p>QmP4FHPbgpnnAdDLZGkSfQD9g7cAGKoZ8X8y3TT1STrgCf is the hash for ipfs-data/food/pizza.jpg.</p><p>We can see that both pizza.jpg files have the same hash, that’s because we use content base addressing instead of Location-based, and since both pictures have the same content, they will have the same hash.</p><p>Let’s get our IPFS object for the entire folder:</p><pre>ipfs object get QmQG1gw59EhQ1PSTxjYrQuv5xhsWBpqwyH5VuBcVQZCxC6 |jq</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kGDSPlR1mrk0c5gT.png" /></figure><p>Let’s see in-depth how our chunks are split and links between them, for that we will use graphmd.</p><pre>graphmd QmbFrXkhkYrgVSTTi6C4GbCofiyuDf2f8tMkxwrnyfHyfD | dot -Tpng &gt; tree.png</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ycb5jatebfKAv9jP.png" /></figure><p>From the previous schema, we can see that both pizza.jpg files have links pointing to the same Hash, which confirms that they have the same content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lKgrzf6lOpBarSw6.png" /></figure><p>Now let’s navigate and see our picture space.jpg which has the hash: <strong>QmQkwvTvdtxVt9H51MidmhQJqXDEDSa3roT3Tr6Srso9pG</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xw4h2B4FWQS1RDNu.png" /></figure><pre>ipfs object get QmQkwvTvdtxVt9H51MidmhQJqXDEDSa3roT3Tr6Srso9pG | jq</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SNJZQzRLVtb0x_wg.png" /></figure><p>so space.jpg was split into 5 chunks of 256kb and 142kb in the 6th chunk.</p><p>Let’s get only the first chunk with hash <strong>QmNuDn3azR2mQQZ4e77NU6qwZgJ3dLbV768fPopLGgdGDh</strong> using ipfs cat and save it into space-part1.jpg:</p><pre>ipfs cat QmNuDn3azR2mQQZ4e77NU6qwZgJ3dLbV768fPopLGgdGDh &gt; space-part1.jpg</pre><p>when we visualize space-part1.jpg, we can see a small part of the image, and the rest is blank or void:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DTgTaPzYJZUu9EMb.png" /></figure><h3>The IPFS Cluster</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ozBJzbN68wJv-jax" /></figure><p>IPFS Cluster does not act as a traditional concept of the cluster because all IPFS peers are independent and the IPFS cluster software is syncing the data through a Consensus protocol.</p><p>IPFS Cluster provides data orchestration across a swarm of IPFS daemons by allocating, replicating, and tracking a global pin-set distributed among multiple peers.</p><p>IPFS Cluster uses a demon called <em>ipfs-cluster-service</em> that runs as an independent service that interacts with the IPFS daemon’s API.</p><p>Handle replication of millions of pins to hundreds of IPFS daemons in a <em>fire&amp;forget</em> fashion: pin lifetime tracked asynchronously</p><p>Cluster peers take care of asking IPFS to pin CIDs at a sustainable rate and retry pinning in case of failures.</p><p>The cluster comes with a fully-featured API and CLI, <em>ipfs-cluster-ctl</em> provides a command-line client to the fully-featured Cluster HTTP REST API.</p><p>The cluster peers form a distributed network and maintain a global, replicated, and conflict-free list of pins.</p><p>The cluster also comes with built-in permissions, providing an embedded permission model that supports standard peers (with permissions to change the cluster pin-set) and follower peers (which store content as instructed but cannot modify the pin-set).</p><p>Note that every pin supports custom replication factors, names, and any other custom metadata.</p><h3>What is Bitswap?</h3><p>Bitswap is a core module of IPFS for exchanging blocks of data. It directs the requesting and sending of blocks to and from other peers in the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/332/0*2_znVqirfXCMde6y" /></figure><p>Bitswap is a <em>message-based protocol</em> where all messages contain <a href="https://docs.ipfs.io/concepts/bitswap/#want-list">want-lists</a> or blocks. Bitswap has a <a href="https://github.com/ipfs/go-bitswap">Go implementation</a></p><p><strong>Bitswap has two main jobs:</strong></p><ul><li>Acquire blocks requested by the client from the network.</li><li>Send blocks in its possession to other peers who want them.</li></ul><h3>Misunderstandings!</h3><p>IPFS is not a place you upload to, it’s a network you’re a part of, meanwhile putting something into IPFS will magically make the Internet host the files forever.</p><p><strong>IPFS is the permanent web: </strong>You shouldn’t read “permanent” as “eternal”, but as “not temporary”, in the same sense as “permanent address” or “permanent employment”.<br>HTTP is <em>temporary</em> in the sense that you’re relying on a single server to <em>actively</em> continue serving a file for it to remain available. IPFS is <em>permanent</em> in the sense that, so long as the file exists somewhere in the network, it will remain available to everyone.#</p><p><strong>IPFS is web 3.0 and will replace HTTP:</strong> The first being that no user would want to set up a full node. People on the IPFS network now are tech enthusiasts, however, no user will want to deal with hashes.</p><p><strong>Hosting a website on IPFS: </strong>Hosting a website on IPFS using IPNS is simply terrible, and not user-friendly at all. Even all nodes pin my site, A CDN service such as Cloudflare would be way more effective and efficient.</p><p><strong>IPFS for private data: </strong>Though of course there’s no reason you can’t store encrypted data on IPFS, encrypted data can’t be de-duplicated and it can’t be intelligently cached, because only its owner knows what it is.</p><p>The result is that accessing encrypted data on IPFS is not much more efficient than HTTP with some simple caching.</p><ul><li>Guaranteed availability</li><li>Performance</li><li>Security</li></ul><h3>Do peers tell what I have?</h3><p>Peers who request content being advertised from a node can retrieve it and see that the node had that content.</p><ul><li>Every IPFS node on the network has a unique public identifier, known as a <strong>PeerID</strong>.</li><li>Since PeerIDs are long-lived identities, it’s possible for someone to look up your PeerID in the distributed hash table (DHT) that stores public IPFS metadata.</li><li>DHT contains metadata about the content that is requested or provided by PeerIDs,</li></ul><h3>Cost of the IPFS Cluster</h3><p>Storage on the blockchain is extremely expensive.</p><p><strong><em>Replication:</em> </strong>We want to have a replication factor of 3, meaning at least 3 copies of each transaction on 3 distinct nodes.</p><p>We are going to use <a href="https://www.scaleway.com/en/">Scaleway</a> as our cloud provider.<br>To provision a private IPFS Cluster for blockchain purposes, with 3 nodes we are going to need :</p><ul><li>3 VMs as nodes</li><li>Private Load balancer</li><li>Public Load balancer</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kei4Zo7mtIWEfL1O" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y5pWizE7sawiKG3c" /></figure><p><strong>So the storage cost of 1TB will cost 99.25 EUR and 255 EUR for the infra monthly.</strong></p><h3>Security risks</h3><p>Do not consume websites hosted on IPFS via path-based gateways, as this completely bypasses the same-origin policy.</p><p>Do not get misled by a shiny ✅ on the public<a href="https://ipfs.github.io/public-gateway-checker/"> IPFS gateway checker list</a>.</p><p>The ✅ next to CORS means that the gateways wildcard allows all CORS access. <strong>This can be a security issue.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6SNXafhnoreIMDt4" /></figure><p>Be aware that public gateways may default to enabling CORS for all domains, allowing one CID to interact with resources from another CID (and potentially data stored for that CID).</p><p><strong>You are trusting the gateway to deliver the content you requested.</strong> -&gt; There is no way for your browser to verify whether the content received is what is stored in the IPFS distributed filesystem.</p><ul><li>You are trusting the gateway to not alter the CID before delivering it to you. Therefore, we can only <strong>recommend running a self-hosted gateway</strong>.</li><li>The gateway may keep a protocol of your connection details and the data you have requested.</li></ul><h3>Major IPFS Users</h3><ul><li><a href="https://developers.cloudflare.com/distributed-web/ipfs-gateway"><strong>Cloudflare</strong></a><strong>, </strong>runs a distributed web gateway to speed up and secure access to IPFS without the need for a local node.</li><li><a href="https://techcommunity.microsoft.com/t5/identity-standards-blog/ion-booting-up-the-network/ba-p/1441552"><strong>Microsoft ION</strong></a>, The digital identity system is built on Bitcoin and IPFS and its objective is to build a technology that allows creating a secure and scalable digital identity system at a global level.</li><li><a href="https://brave.com/"><strong>Brave,</strong></a> the search engine uses Origin Protocol and IPFS to host its decentralized merchandise store.</li><li><a href="https://blog.ipfs.io/2020-03-30-ipfs-in-opera-for-android/"><strong>Opera para Android</strong></a> has default support for IPFS, allowing mobile users to navigate ipfs: // links to access data on the IPFS network.</li><li><a href="https://github.com/ipfs/distributed-wikipedia-mirror"><strong>Wikipedia use IPFS</strong></a> They have developed a mirror of their website, which allows them to access Wikipedia from jurisdictions where it is censored.</li><li><a href="https://filecoin.io/"><strong>Filecoin</strong></a>, uses IPFS to create a cooperative storage cloud based on IPFS.</li></ul><h3>Deploying Kalvad’s Website on IPFS Network</h3><pre>ipfs add -r ./sample_website<br>added QmaYZE5QQVR3TqQSL5WDcwuxUwBqczbMkmbHeoeNEAumbe sample_website/index.cssadded Qme5TzVW4en8HWP49N9nLHwUkAMhCn6zvWteoknmFDhwJg sample_website/index.htmladded QmThzr7LgYmUz6tfTArJK5tbrtidXSA12ZCL4ESy6HB8P2 sample_website/index.jsadded QmSYDncaMte6GPx5jC7kqeXWnrKsCp4nkLfQXPxiAKtECU sample_website/something.htmladded QmQ6HiGM7ikKgdyVuzfL7VukzaMCyxtNRnsnPu4nTPG9X3 sample_website 1.01 KiB / 1.01 KiB [=================================================] 100.00%</pre><p>The hash printed in the last line is the CID for the whole directory and thus our website. We can see the sample website hosted on<br><a href="https://ipfs.io/ipfs/QmeUG2oZvyx4NzfpP9rruKbmV5UNDmTQ8MoxuhTJGVZVTW/">https://ipfs.io/ipfs/</a>QmUCF47VHN8PfUVjYfeL2MWUaeKN2twrQSfi8i4Shcu8dz</p><p>We have already done it, please find Kalvad’s website in the IPFS network :</p><p><a href="https://ipfs.io/ipfs/QmUCF47VHN8PfUVjYfeL2MWUaeKN2twrQSfi8i4Shcu8dz/">https://ipfs.io/ipfs/QmUCF47VHN8PfUVjYfeL2MWUaeKN2twrQSfi8i4Shcu8dz</a></p><h3>Conclusion</h3><p>In this post, We covered what IPFS is, how it works, and what was the reason behind its creation, and at what cost.</p><p>Whether you are a fan of IPFS or not, it’s is an ambitious vision of a new decentralized Internet infrastructure, where it can be used in many different areas such as Blockchain.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=87cfef633f2b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/ipfs-in-depth-87cfef633f2b">IPFS in-depth</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to compile binutils on Ubuntu/Debian. (Upgrade or Downgrade)]]></title>
            <link>https://soufianebouchaa.medium.com/how-to-compile-binutils-on-ubuntu-debian-ec268674b759?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec268674b759</guid>
            <category><![CDATA[downgrade-binutils]]></category>
            <category><![CDATA[upgrade-binutils]]></category>
            <category><![CDATA[binutils]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Tue, 01 Mar 2022 07:50:04 GMT</pubDate>
            <atom:updated>2022-03-01T07:52:31.758Z</atom:updated>
            <content:encoded><![CDATA[<p>If you are facing an issue that looks like the following:</p><pre>error: linking with `cc` failed: exit status: 1</pre><pre>= note: /usr/local/bin/ld: BFD (GNU Binutils) 2.35 assertion fail elflink.c:14788</pre><pre>collect2: error: ld returned 1 exit status</pre><p>Then you should check your binutils version, by running the following command:</p><pre>$ ld -v :</pre><pre>GNU ld (GNU Binutils) 2.35</pre><p>To downgrade/Upgrade Binutil version ,​ you will need first this package for the compiling process:</p><pre>sudo apt-get install texinfo</pre><p>Full list of binutils version available here :</p><p><a href="http://ftp.gnu.org/gnu/binutils/">http://ftp.gnu.org/gnu/binutils/</a></p><p><strong>Let’s pick binutils 2.34,</strong></p><pre>wget <a href="http://ftp.gnu.org/gnu/binutils/binutils-2.34.tar.gz">http://ftp.gnu.org/gnu/binutils/binutils-2.34.tar.gz</a></pre><pre>tar -xf binutils-2.34.tar.gz</pre><pre>cd binutils-2.34</pre><p>Then we will run the configure bash script, to be installed at /usr/local :​</p><pre>./configure — prefix=/usr/local</pre><p>Then</p><pre>make</pre><pre>sudo make install</pre><p>let’s check now​, make sure its<strong> 2.34</strong></p><pre>ld -v</pre><p>Also Make sure its <strong>/usr/local/bin/ld</strong></p><pre>which ld</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec268674b759" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monitoring Production from A to Z, this is your CrashPlan]]></title>
            <link>https://blog.devgenius.io/monitoring-your-production-from-a-to-z-this-is-your-crashplan-3e727d6ec04c?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e727d6ec04c</guid>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[production]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[influxdb]]></category>
            <category><![CDATA[grafana]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 25 Sep 2021 07:45:46 GMT</pubDate>
            <atom:updated>2021-09-25T10:51:47.873Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IxGVOc_6YvOXCpf9c5RSbA.png" /><figcaption>Photo by <a href="https://unsplash.com/@jaqbovsky?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Michał Jakubowski</a> on <a href="https://unsplash.com/s/photos/monitoring?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Your monitoring plan can be split into three major parts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xh0O6f1XYg9Nctl9e67r9Q.png" /><figcaption>Observability, Monitoring, and Analysis combine to provide actionable intelligence.</figcaption></figure><p><strong>1- Observability</strong> Observability enables us to collect all logs and events and behaviors of the servers.</p><p><strong>2- Monitoring</strong> provides feedback from production and delivers information about an application’s performance and usage patterns, such as :</p><ul><li>Tells you whether the system works</li><li>A collection of metrics and logs about a system</li><li>Failure-centric</li><li>Is “the how” / Something you do</li></ul><p><strong>3- Observability</strong></p><ul><li>Lets you ask why it’s not working</li><li>The dissemination of information from that system</li><li>Understands system behavior regardless of an outage</li><li>Is “the goal” / Something you have</li><li>You make yourself observable</li></ul><h3>Observability vs Monitoring? clear the confusion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n07pedsJeuyMLp8BI2OtFw.png" /></figure><h3>Let’s make our hands dirty</h3><p>Let’s make the CrashPlan for monitoring a company called ‘test’</p><p>Imagine that we have a running website, that has multiple services, each service is hosted with a different subdomain.</p><p>The complete schema for the company is the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXg15neQAcBrM2BCdM1LBQ.png" /></figure><p>To achieve perfect monitoring, we need to list down all the dependencies:</p><p><strong>the store</strong> and <strong>service.test.com</strong> and <strong>the landing page</strong> and <strong>the mobile apps</strong> depend on the API.service.com (4 dependencies)</p><p>The <strong>API.service.com</strong> depends on <strong>the 3rd Party API.</strong></p><p>So, It means that we have 2 important crash points, or specifically we call it S<strong>ingle Point of Failure.</strong></p><p>Unfortunately, we cannot change the structure of the company, but we need to provide good monitoring and alerting in order to increase the availability percentage of the service.</p><p><strong>— Monitoring with Monit:</strong></p><p><strong>Monit</strong> is a utility for managing and monitoring processes, programs, files, directories, and filesystems on a Unix system.</p><p>Monit is running in daemon mode, it is configured by <strong>cycles </strong>which is a polling interval in seconds<strong>.</strong></p><p>In our example, we are defining 1 cycle = 5 seconds</p><p><strong>A few Monit Rules, that can be very helpful:</strong></p><ul><li>Loadavg (15min) &gt; vCPU x for 12 cycles</li><li>CPU usage &gt; 90% for 12 cycles</li><li>Memory usage &gt; 75% for 12 cycles</li><li>Uptime &gt; 90 days</li><li>Disk usage &gt; 80%</li><li>Disk usage &gt; 90%</li><li>DB Service should be UP for 1 cycles</li><li>SSH Service should be UP (Port 22) for 6 cycles</li><li>Monitor Login page by simulating HTTP Requests to the API (and Expect a status code of 200 or raise an alert)</li><li>Monitor reachability of all critical endpoints monitor the page speed load</li><li>Monitor the SSL Validity should be treated for 2 weeks, which gives you some time to breathe and plan for an SSL upgrade.</li><li>Monitor the Latency between different components running.</li><li>Monitor all the external and 3rd Party endpoints</li></ul><p>Once an alert is detected, a trigger should be fired to inform the Ops team :</p><p><strong>Rules triggers</strong></p><ul><li>The alert should be recorded for reference on Alerta Dashboard with the correspondent severity level.</li><li>A slack notification should be sent to a Slack Channel/ Discord/ email. etc</li></ul><p>Please find the complete documentation of Monit here: <a href="https://mmonit.com/monit/documentation/monit.html">https://mmonit.com/monit/documentation/monit.html</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/879/0*sag2auPJR3n-BjNN.png" /><figcaption>Monit Dashboard that should be deployed on every server</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YTNRra5EEUfUqoN3.png" /><figcaption>Alerts Dashboard on alerta GUI</figcaption></figure><p>Please find the complete documentation of Alerta here:<br><a href="https://docs.alerta.io/en/latest/">https://docs.alerta.io/en/latest/</a></p><p><strong>— Monitoring performance Elastic APM :</strong></p><p>An APM is required to install it on the API and the shop and the other service to monitor, detect, and diagnose complex application performance issues to maintain an expected level of service</p><ul><li>Provide visibility</li><li>Identify root causes</li><li>Maintain service levels</li><li>Scale with growth</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gotU6a6ysBCs2abZ.png" /></figure><p>Elastic APM is available for most of the languages :</p><ul><li><a href="https://www.elastic.co/guide/en/apm/agent/go/current/index.html">APM Go Agent [1.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/go/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/swift/current/index.html">APM iOS Agent [0.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/swift/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/java/current/index.html">APM Java Agent [1.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/java/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/dotnet/current/index.html">APM .NET Agent [1.11]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/dotnet/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/nodejs/current/index.html">APM Node.js Agent [3.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/nodejs/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/php/current/index.html">APM PHP Agent [1.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/php/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/python/current/index.html">APM Python Agent [6.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/python/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/ruby/current/index.html">APM Ruby Agent [4.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/ruby/index.html">other versions</a></li><li><a href="https://www.elastic.co/guide/en/apm/agent/rum-js/current/index.html">APM Real User Monitoring JavaScript Agent [5.x]</a> — <a href="https://www.elastic.co/guide/en/apm/agent/rum-js/index.html">other versions</a></li></ul><p><strong>— Server Metrics with TICK Stack :</strong></p><p>We use telegraf to have a minimal memory footprint of the hardware metrics for upcoming debugging.</p><p>We use influx DB, a time series DB to store all the real-time metrics,</p><p>We use Grafana to visualize all the data and create wonderful Dashboards</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uXLsYq4FfQeHPQ7f" /><figcaption>Example of a Dashboard in Grafana</figcaption></figure><p>Full documentation for telegraf : <a href="https://www.influxdata.com/time-series-platform/telegraf/">https://www.influxdata.com/time-series-platform/telegraf/</a></p><p>Full documentation for influxDB: <br><a href="https://docs.influxdata.com/influxdb/v2.0/">https://docs.influxdata.com/influxdb/v2.0/</a></p><p><strong>— Error reporting with Sentry:</strong></p><p>Sentry is cross-platform application monitoring, with a focus on error reporting.</p><p>Where you can track errors on the running apps of your production and resolve by pushing new releases as soon as they happen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WhTY4D8x-lrGbu6p.png" /><figcaption>Sentry Dashboard of Production errors.</figcaption></figure><p>I highly recommend deploying the self-hosted version of sentry because it’s free and easy to deploy.</p><p>Full documentation: <a href="https://develop.sentry.dev/self-hosted/">https://develop.sentry.dev/self-hosted/</a></p><p>I hope that I came across all the pain points of monitoring, alerting, and Observability.</p><p>If you have any questions or you need more details, please feel free to contact me at contact@soufianebouchaara.com</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e727d6ec04c" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/monitoring-your-production-from-a-to-z-this-is-your-crashplan-3e727d6ec04c">Monitoring Production from A to Z, this is your CrashPlan</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated testing for GitHub actions & Newman automation from Postman]]></title>
            <link>https://soufianebouchaa.medium.com/automated-testing-for-github-actions-newman-automation-from-postman-e50b9c923692?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/e50b9c923692</guid>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[automation-testing]]></category>
            <category><![CDATA[postman-collections]]></category>
            <category><![CDATA[ci-cd-pipeline]]></category>
            <category><![CDATA[quality-assurance]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Sat, 04 Jul 2020 14:13:15 GMT</pubDate>
            <atom:updated>2020-07-04T14:24:44.119Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*NxiTPnchg_NEYrBPqdBrHw@2x.jpeg" /></figure><p>Newman is a command-line runner for Postman collections. In other words, it allows a user to run an existing Postman collection through the command line.</p><p>Newman is a NodeJS module , for that we need to have nodes installed, to check, simply check the node version on the system using the below command:</p><blockquote>Node -v</blockquote><h3>Installation</h3><p>You can simply install Newman like any node package using the following command:</p><blockquote>npm install -g newman</blockquote><p>When it’s installed, you can check its existences:</p><blockquote>newman -v</blockquote><h3>Running Collections Using Newman</h3><p>To run collections using Newman, you should have any one of the 2.</p><ul><li>The collection in JSON format.</li><li>URL of the hosted collection.</li></ul><p>The command used to run the Postman collection using Newman is:</p><blockquote>newman run {{collectionJsonPath}}</blockquote><p>OR</p><blockquote>newman run {{collectionUrl}}</blockquote><p>Let’s try running a sample collection using Postman.</p><p>Go to the Postman application, use any existing Postman collection and export it to JSON form.</p><p>Now export the collection to JSON format (collection.json)</p><p>Once the collection JSON is obtained, open the terminal and run the collection using Newman run command</p><blockquote>newman run collection.json</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/617/1*P1CKrJB_fMV5vSvrKc4p_Q@2x.jpeg" /></figure><p>Once the above command is run, the following output is produced.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*peSrWdNEwLe0Hl8EVoswgg@2x.jpeg" /></figure><h3>GitHub Actions</h3><p>GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub.</p><p>In order to run a GitHub Actions workflow automatically we need to create github actionb workflow folder and files.</p><h3>GitHub Action workflow file</h3><p>First, we need to setup a new workflow file in GitHub Action Workflow folder, which listens for the repository dispatch event.</p><p>More informations please visit: ttps://github.com/features/actions</p><p><a href="https://docs.github.com/en/actions">GitHub Actions documentation - GitHub Docs</a></p><h3>Workflow File Syntax</h3><p>Workflow files use YAML syntax, and must have either a .yml or .yaml file extension. If you’re new to YAML and want to learn more, see “<a href="https://www.codeproject.com/Articles/1214409/Learn-YAML-in-five-minutes">Learn YAML in five minutes.</a>”</p><p>Example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*1gyxOZsl1uOw_CdqFnIAhg@2x.jpeg" /></figure><p>At the moment of writing this article, the newman action is on pre-release and here’s the repository for it :</p><p><a href="https://github.com/marketplace/actions/newman-action">Newman Action - GitHub Marketplace</a></p><p>As described on their README you can run it in two different ways :</p><ul><li>Via URL</li><li>Via Potman API</li></ul><p>Passing these IDs along with your Postman API key will run Newman in this method.</p><p><a href="https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners#creating-and-using-secrets-encrypted-variables">See Creating and using secrets</a> for how to add your Postman API to GitHub securely.</p><h3>Using Newman and github actions</h3><p>lets create a new worflow file in the following path:</p><p>/.github/worflow/newman-github-actions.yml</p><p>please find the workflow file for this test automation that’s gtriggered when code is pushed to master or staging branches:</p><p><a href="https://gist.github.com/soufiane-bouchaara/358fafe93df6b44d2f9bd86210197043">https://gist.github.com/soufiane-bouchaara/358fafe93df6b44d2f9bd86210197043</a></p><p>if you have any question or sugesstioon please feel free to leave a comments</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e50b9c923692" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[It’s never too late to change your career to DevOps if you’re a Software Engineer]]></title>
            <link>https://blog.devgenius.io/its-never-too-late-to-change-your-career-to-devops-if-you-re-a-software-engineer-59596d27246e?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/59596d27246e</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[career-development]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Fri, 12 Jun 2020 08:14:53 GMT</pubDate>
            <atom:updated>2020-07-18T23:14:13.545Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/405/1*8FPPfbbsFKNInw8e2dQ6AA@2x.jpeg" /></figure><p>My name is Soufiane Bouchaara, I’m a Lead DevOps Engineer with +5 years experience in software engineering, starting with PHP, C#.NET to Javascript, Python, Go, and a bit of Rust.</p><p>I’ve done both Dev and Ops in my career and found the division almost impossible from both sides. Long before “DevOps” was a named thing, engineers like me were working to tear down that wall and provide enablement through automation.</p><p>Switching my mission to DevOps was a very challenging step, because it’s about a new culture and not just coding.</p><p>It’s bout maintaining the integrity of the product, site reliability engineering by wearing a lot of different hats and shift focus constantly, depending on incoming fires</p><p>I have accumulated a concrete and successful experience in implementing and adapting SLOs(Service Level Objectives) to examine what is the level of the reliability of service, in order to improve it. Moreover, Cost killing is also one of my strengths.</p><p>It’s about a strange feeling when you become a Senior Software Engineer with more than 5 years of experience, you feel like starting from scratch again, but being at this position with a strong background of software engineering enabled me to deep dive into the architectural level of every project that I’m working on and be part of every C-Level decisions, not only a simple Ops used to deploy and monitor.</p><p>Being part of C-Level decisions enabled me to focus on the ROI (Return On Investment) by fastening deployments and automation which decreased time to market and reduce any human intervention,</p><p>Although productivity is more difficult to quantify from a financial standpoint than regular Ops metrics, Lead DevOps should take a big responsibility to improve the return on investment.</p><p>Lead DevOps engineers are successful in their lives if strong backgrounds of programming and framework both is there.</p><p>People coming from any department to DevOps should understand what the developers are doing in the development phase and how they are managing the versions of codes, integrating, deploying to servers. Once they understand how things are done manually with no automation tools, things will get easier for them. In Addition to this, one can’t become expert in DevOps, if he is not good in operations.</p><p>Being able to assist in maximizing my company’s investment and at the same time increase velocity for delivering new and difference-making features while doing so with increased efficiency is deeply gratifying.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59596d27246e" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/its-never-too-late-to-change-your-career-to-devops-if-you-re-a-software-engineer-59596d27246e">It’s never too late to change your career to DevOps if you’re a Software Engineer</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monitor the performance of your NodeJs app using Elastic APM]]></title>
            <link>https://soufianebouchaa.medium.com/monitor-the-performance-of-your-nodejs-app-using-elastic-apm-25c27a0f0f8d?source=rss-61cca0053586------2</link>
            <guid isPermaLink="false">https://medium.com/p/25c27a0f0f8d</guid>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[elastic]]></category>
            <category><![CDATA[apm]]></category>
            <dc:creator><![CDATA[Soufiane Bouchaara]]></dc:creator>
            <pubDate>Thu, 11 Jun 2020 21:19:23 GMT</pubDate>
            <atom:updated>2020-06-11T21:19:23.007Z</atom:updated>
            <content:encoded><![CDATA[<p>To Monitor the performance of Stark, we will need the following stack:</p><p><strong>APM Node.js Agent :</strong> sends performance metrics and errors to the APM Server. It has built-in support for the most popular frameworks and routers.</p><p><strong>APM Server :</strong> receives data from APM agents and transforms them into Elasticsearch documents.</p><p><strong>ElasticSearch</strong>: a database used for storing and indexing logs</p><p><strong>Kibana:</strong> is a visualization tool (a web interface) that uses Elasticsearch DB to Explore, Visualize, and Share</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><h3>The key features of implement an APM</h3><p><strong>Easy Implementation</strong></p><p>The integration of APM with NodeJS — Express app is easy, and there are various ways we can tweak it to fit our needs.</p><p>The only updates we have to add to the coding, is a single call of the APM Agent to run it before everything else.</p><p>Example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><p><strong>Performance of every web request and transaction</strong></p><p>Using an APM will enable us to measure the performance of every web request and transaction in your application.</p><p>We can then use this to understand which requests are accessed the most, which are the slowest, and which ones you should add to your backlog to improve.</p><p><strong>Code level performance profiling</strong></p><p>By tracking what the application is doing all the way down to the code level, we can potentially gain more insights about what is occurring:</p><ul><li>What key methods in your code are even being called?</li><li>Which methods are slow?</li><li>Is the app slow due to things like JIT, garbage collection, etc?</li><li>What dependencies are being called?</li></ul><p><strong>Application errors</strong></p><p>We need to find and fix the errors, or at least know about them, before or customers call to tell us because odds are most of them won’t even call to tell you.</p><p>We can set up alerts for new exceptions as well as for monitoring overall error rates.</p><p>Here are some screenshots of a Kibana Dashboard Connected with Elastic APM</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=25c27a0f0f8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>