<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Riteek Srivastav on Medium]]></title>
        <description><![CDATA[Stories by Riteek Srivastav on Medium]]></description>
        <link>https://medium.com/@riteeksrivastava?source=rss-5b0a5456deaa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*uhx17_U6RFLX9TZedneT8Q.jpeg</url>
            <title>Stories by Riteek Srivastav on Medium</title>
            <link>https://medium.com/@riteeksrivastava?source=rss-5b0a5456deaa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:27:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@riteeksrivastava/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Minimum Detectable Effect(MDE) ]]></title>
            <link>https://riteeksrivastava.medium.com/minimum-detectable-effect-mde-6cf5f255dc1f?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/6cf5f255dc1f</guid>
            <category><![CDATA[ab-testing]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[experimentation]]></category>
            <category><![CDATA[sample-size-calculation]]></category>
            <category><![CDATA[p-value]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Mon, 09 Aug 2021 05:16:05 GMT</pubDate>
            <atom:updated>2021-08-09T05:51:40.370Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Prerequisite: </em><a href="https://medium.com/gojekengineering/introducing-litmus-gojeks-own-experimentation-platform-3803467b6a53"><em>A/B testing</em></a><em>, </em><a href="https://www.surveymonkey.com/mp/sample-size-calculator/"><em>Sample size calculation</em></a></blockquote><p>This short article will explain about what is the MDE with an example and how does it help in sample size calculation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6T2JX-WEBewTsT9s7S_bQ.jpeg" /></figure><p>As the name suggests MDE is the minimum change that you want your experiment/test to detect. It is an important factor in sample size calculation and is inversely proportional to it. <br><strong>Let me explain this with an example</strong>. Suppose you want to test a new drug for dogs that whether it can be used for increasing their height or not. And let’s say there are two cases, where you are saying that:</p><ol><li>the increase in height should be 1cm(MDE).</li><li>the increase in height should be 10cm(MDE).</li></ol><p><strong>In the 1st</strong> case since the change is very small hence to justify that it has not happened by chance, you will need more samples. (In the statistics world, we say you will need a high sample size to make the result statistically sound).</p><p>On the other hand <strong>in the 2nd</strong> case since the proposed change in height is large you can be confident even with the smaller sample size that the change is due to the drug.</p><p>So with the above example, we can say that MDE is used to measure the experiment sensitivity. To detect the smaller change(high sensitivity) you will need a bigger sample size and to detect the bigger change(low sensitivity) you will need a smaller sample size. So mathematically you can say that:</p><pre>MDE = K/Sample-size = K&#39;/sensitivity-of-experiment</pre><p><strong>You can have a doubt</strong> here that why not always have the low MDE for the experiments. The answer is, you can have but… low MDE comes at some cost: you will have to run the experiments longer to reach the bigger sample size, and hence you will have to spend more money to achieve that traffic. Like in the drug example, you may need a bigger infrastructure to handle a larger sample size, and a more precise and accurate instrument to measure that small change.</p><p>MDE is not something which we calculate, we propose or decide MDE for a particular experiment and it can be different for different experiments, like you can see <a href="https://www.brooksbell.com/resource/blog/minimum-detectable-effect/?__cf_chl_managed_tk__=pmd_178c32e2b94cb5c2a592fe03baa8f6beedaf1336-1628425870-0-gqNtZGzNAuKjcnBszQmO">here</a> how the MDE is different for experiments on different pages of a website.</p><p>I hope this will help you understand the MDE better. If you have further questions related to MDE you can ask in the comment section.</p><p>Happy Testing :)</p><p>References</p><ol><li><a href="https://splitmetrics.com/resources/minimum-detectable-effect-mde/">https://splitmetrics.com/resources/minimum-detectable-effect-mde/</a></li><li><a href="https://www.brooksbell.com/resource/blog/minimum-detectable-effect/?__cf_chl_managed_tk__=pmd_178c32e2b94cb5c2a592fe03baa8f6beedaf1336-1628425870-0-gqNtZGzNAuKjcnBszQmO">https://www.brooksbell.com/resource/blog/minimum-detectable-effect/?__cf_chl_managed_tk__=pmd_178c32e2b94cb5c2a592fe03baa8f6beedaf1336-1628425870-0-gqNtZGzNAuKjcnBszQmO</a></li><li><a href="https://www.youtube.com/watch?v=BHGB8WS0XbI&amp;ab_channel=BrianDavidHall">https://www.youtube.com/watch?v=BHGB8WS0XbI&amp;ab_channel=BrianDavidHall</a></li><li>Thanks to <a href="https://medium.com/u/41287746e0ad">Divyatman Khare</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6cf5f255dc1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DNS (Domain Name System)]]></title>
            <link>https://riteeksrivastava.medium.com/dns-domain-name-system-c91bdb36b3ba?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c91bdb36b3ba</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[computer-network]]></category>
            <category><![CDATA[dns]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Sun, 11 Apr 2021 18:30:21 GMT</pubDate>
            <atom:updated>2021-04-11T18:30:21.126Z</atom:updated>
            <content:encoded><![CDATA[<p>In this blog I will be explaining about the basics of DNS which will include:</p><ul><li><strong>What is DNS?</strong></li><li><strong>What services does it provide?</strong></li><li><strong>How does it work?</strong></li><li><strong>Different types of DNS records.</strong></li></ul><h3>What is DNS</h3><p>Humans can have different identifiers<em>(name, driving license number, PAN etc)</em>. Within a context, one identifier can be more appropriate than others. Like humans prefer to use “name” identifier, because they are more convenient to remember and refer to. Similarly in networks, hosts can have different identifiers. One of them is hostnamesuch as google.com, amazon.in, etc. Hostnames are easy to remember and refer hence are appreciated by humans.<br>But the variations in the hostname <em>(can be of variable lengths, can have alphanumeric characters)</em> make it difficult for machines<em>(routers and other network devices)</em> to process it. And for the above reason hosts are also identified by <a href="https://www.kaspersky.com/resource-center/definitions/what-is-an-ip-address">IP addresses</a> <em>(more rigid structure)</em>.</p><p>The different preferences for host identifiers demand for a directory service in the internet which translates the hostnames to IP addresses. And this is the main task of<strong> </strong>the<strong> domain name system(DNS). </strong>So in short form DNS is a distributed database in the internet which stores the record for hostnames translations.</p><h3>Services provided by DNS</h3><ul><li><strong>Host Resolution:</strong> This is the main task of DNS. Suppose you hit the url <a href="http://www.google.com/">www.</a>samplehost.com/index.html. For sending the http request from your computer to the web server <a href="http://www.samplhost.com/">www.samplehost.com</a>, your computer first obtain the IP address of <a href="http://www.sample.com./">www.samplehost.com.</a> The DNS client running on your computer queries the DNS server for the resolution of hostname <a href="http://www.samplehost.com/">www.samplehost.com</a> , and eventually receives the IP address. Then your computer initiates the TCP connection with resolved IP address.</li><li><strong>Host aliasing:</strong> Some hostnames can be complicated and hence can have one or more alias name. e.g. disco.dance.entertainment.com can have two aliases such as entertainment.com and <a href="http://www.entertainment.com./">www.entertainment.com.</a> In this case disco.dance.entertainment.com is known as canonical hostname. DNS can be invoked to obtain the canonical hostname for the supplied alias as well as the IP address of the host.</li><li><strong>Load distribution: </strong>DNS can also be used for load distribution among replicated servers. The sites with heavy traffic or loads are replicated over multiple servers for high availability and other some other reasons<em>(out of the scope of this blog)</em>. In this case when client makes query to the DNS server, it returns the set of IP addresses and rotates the ordering of IP addresses in each reply.</li><li><strong>Mail Server aliasing</strong>: We have observed that the email ids are like blog@gmail.com , blog@outlook.com etc. However, the hostname of the gmail or outlook servers are more complicated than gmail.com or outlook.com and are less convenient to remember<em>(something like `xyz.nyc.dom.gmail.com` and `abc.cad.dom.outlook.com`)</em>. So in this case DNS server can be invoked by mail application to obtain the canonical hostname(`<em>xyz.nyc.dom.gmail.com`</em>) for the alias hostname (gmail.com)as well as IP address.</li></ul><h3>How does DNS works?</h3><p>Whenever an application on a user’s machines needs to translate a hostname to IP, it invokes the client side of DNS with the hostname that needs to be translated. Then DNS in the user’s host send the query in the network and after some delay(in ms or secs) it receives a DNS reply with the mapping of hostname to the IPs. Finally this mapping is passed to the invoking application.</p><p>From the perspective of invoking application, DNS is a simple translation service which is what most of people think about DNS.</p><p>Now we will see how does this translation happens. What could be a simple design of this service; we can have a single DNS server which contains all the mappings and the clients which needs any translations should query to this server directly. But in todays era with such a high volume of requests in the internet and with such a huge growing number of hosts, this design will not work. <br>Problems with this design are:<br>1. Single point of failure.<br>2. Latency issues. (As a server near to one part of world will be far for from other part of the world)<br>3. Huge Request Volume.</p><p>In order to deal with these issues the design of DNS has to be distributed in nature. DNS uses large number of servers which are distributed around the world and so are the mapping of the hosts.</p><p>There are three classes of DNS:<em> Root DNS Server, Top-Level Domain(TLD) DNS Server and Authoritative DNS server o</em>rganised in a hierarchy as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FUpoVdF8Pf4_UyUfvqQsoA.png" /><figcaption>Image is inspired from the book Compute Networks: Top Down approach</figcaption></figure><p>Let me explain these classes with an example. Suppose a client wants the IP address of <a href="http://www.amazon.in/">www.amazon.in</a>. <br><strong>1.</strong> Client will first connect to one of the root DNS server which will return the IP address of one of the TLD DNS server for domain in .<br><strong>2.</strong> Client will then connect to one of the TLD DNS server which will return the IP address of one of the authoritative DNS server for amazon.in .<br><strong>3.</strong> Finally client will connect to one of the authoritative DNS server of amazon.in which will return the IP address of www.amazon.in .</p><p><em>Generally there is another DNS server comes in picture known as </em><strong><em>local DNS server </em></strong><em>which is provided by ISPs.(Internet server providers). So clients generally talks to local DNS servers and it’s the responsibility of local DNS server to talk to the respective other DNS servers(root, tld, authoritative) to resolve the hostname. Local DNS server also caches the results with some TTL which can be used in further queries.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4pWfeoiVAxmdB6vRcn1I7w.png" /><figcaption>DNS look up via different class of DNS servers</figcaption></figure><h3>Types of DNS records.</h3><p>DNS stores the mappings in form of resource records(RR) which are represented by four tuple which contains the following fields</p><p>(Name, Value, Type, TTL)</p><p>There are four types of DNS recored.</p><ol><li>ForType=A ,Name is the hostname and Value is the IP address of the hostname. e.g. (www.random.com, 143.60.76.7, A, 10)is Type A record.</li><li>ForType=NS ,Name is the domain (such as amazon.in) and Value is the hostname of the authoritative DNS server that knows how to obtain the IP address for the hosts in the domain. e.g (amazon.in, dns.amazon.in, NS, 10) is a type NS record.</li><li>For Type=CNAME , Name is the aliased hostname and Value is the canonical hostname for that aliased hostname. e.g (entertainment.com, disco.dancer.entertainment.com, CNAME, 10)is Type CNAME record.</li><li>For Type=MX , Name is the aliased hostname for mail server and Value is the canonical hostname of that aliased mail server. e.g.(entertainment.com, mail.disco.dancer.entertainment.com, MX, 10) . Point to note here is that by using MX record a company can have the same aliased name for mail server and for some other server, say it’s company’s website. For e.g. google.com can be a hostname for a website as will as hostname for a the google mail server, this can be distinguished using the type of DNS record.</li></ol><blockquote><em>You can use </em><em>nslookup or </em><em>dig commands on linux to check these different type of records. Like:</em></blockquote><pre>nslookup -type=A google.com</pre><pre>Server:  172.20.10.1<br>Address: 172.20.10.1#53</pre><pre>Non-authoritative answer:<br>Name: google.com<br>Address: 142.250.194.174</pre><pre>____________________________________________________________________<br>nslookup -type=MX google.com</pre><pre>Server:  172.20.10.1<br>Address: 172.20.10.1#53</pre><pre>Non-authoritative answer:<br>google.com mail exchanger = 50 alt4.aspmx.l.google.com.<br>google.com mail exchanger = 30 alt2.aspmx.l.google.com.<br>google.com mail exchanger = 20 alt1.aspmx.l.google.com.<br>google.com mail exchanger = 10 aspmx.l.google.com.<br>google.com mail exchanger = 40 alt3.aspmx.l.google.com.</pre><pre>____________________________________________________________________nslookup -type=CNAME google.com</pre><pre>Server:  172.20.10.1<br>Address: 172.20.10.1#53</pre><pre>Non-authoritative answer:<br>*** Can&#39;t find google.com: No answer</pre><pre>Authoritative answers can be found from:<br>google.com<br> origin = ns1.google.com<br> mail addr = dns-admin.google.com<br> serial = 367541562<br> refresh = 900<br> retry = 900<br> expire = 1800<br> minimum = 60<br>____________________________________________________________________nslookup -type=NS google.com</pre><pre>Server:  172.20.10.1<br>Address: 172.20.10.1#53</pre><pre>Non-authoritative answer:<br>google.com nameserver = ns4.google.com.<br>google.com nameserver = ns1.google.com.<br>google.com nameserver = ns2.google.com.<br>google.com nameserver = ns3.google.com.</pre><pre>Authoritative answers can be found from:</pre><p>References: <a href="https://www.amazon.com/Computer-Networking-Top-Down-Approach-7th/dp/0133594149">Computer networks Top down Approach</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c91bdb36b3ba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing Litmus: GOJEK’s Own Experimentation Platform]]></title>
            <link>https://medium.com/gojekengineering/introducing-litmus-gojeks-own-experimentation-platform-3803467b6a53?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/3803467b6a53</guid>
            <category><![CDATA[experiment]]></category>
            <category><![CDATA[gojek]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[a-b-testing]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Thu, 27 Jun 2019 04:04:42 GMT</pubDate>
            <atom:updated>2019-06-27T04:06:38.137Z</atom:updated>
            <content:encoded><![CDATA[<h4>An overview of how we did feature rollouts at GOJEK, and the framework we built to make it better.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iNUdV3tFfPBlKec6TLFOoA.jpeg" /><figcaption><a href="http://bit.ly/2RAeKGm">gojek.jobs</a></figcaption></figure><p>The GOJEK <a href="https://play.google.com/store/apps/details?id=com.gojek.app&amp;hl=en">app</a> has been downloaded over 130 million times in Indonesia, and millions use it every day. In order to cater to the needs of our users, we release new features at a rapid pace. However, when you have such a massive audience base, it becomes difficult to gauge what new features will be useful and make genuine impact. So how do we approach the problem?</p><blockquote>As Steve Jobs famously said, “People don’t know what they want until you show it to them.”</blockquote><p>The best way to approach this then is to release multiple variations of a feature and then decide what works based on the data collected. <strong>In the software world, this is what we call experimentation or </strong><a href="https://en.wikipedia.org/wiki/A/B_testing"><strong>A/B testing</strong></a><strong>.</strong></p><p>To help us do this, we built an experimentation platform.</p><p><strong>TL;DR<br></strong>This post provides an overview of Litmus, GOJEK’s experimentation platform for quick feature tests and roll-out to production.</p><h3><strong><em>Our earlier approach</em></strong></h3><p>Initially, we used to categorise users into segments. If you want to roll out a feature for some users, you had to put them into a particular segment, and gradually add/remove users to it depending on the feedback from the experiment. For e.g. creating a segment for alpha users for feature rollout only to the internal alpha users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*_mBiOL2ilvNzKl-h4_aM0w.png" /></figure><p>The segmentation service stores the mapping of users to segments. Internal services and the consumer app hit this service with a userID and it will return all the segments for that user.</p><blockquote>But, what are these segments? 🤔</blockquote><p><strong><em>Segments can be visualised as a tag present on userID.</em></strong> In the consumer app context, whenever a feature is rolled out, the product team can choose to roll it out to ‘pre-tagged’ segments of users like <strong>power, medium,</strong> and <strong>low</strong> users of that product. (These segments can be defined depending on a person’s behaviour on the GOJEK app).</p><p><strong>This approach did cause some problems, though:</strong></p><ol><li><strong><em>Adding and removing users to segments:</em></strong><em> </em>Whenever we release a feature, the general convention is that we roll it out to an increasing percentage of users while monitoring crashes and conversions at every phase. The problem here is that we need to manually add additional users to the respective segments at each of the above rollout phases, which is not very efficient.</li><li><strong><em>Roll out to 100% users</em>:</strong> Say after x% of the rollout with a segment we are confident that the feature needs to be rolled out to 100% users. Here, we have two options: either we put all the existing users to that particular segment and keep adding new users to that (which isn’t a great approach for a large user base) or just get rid of the segmentation check from the consumer app side. If we go with the second approach, it will require a fresh app release on the store and usually, users choose to remain on older versions of the app until the version isn’t supported anymore.</li><li><strong><em>Dynamic Segments:</em></strong><em> </em>What if someone wants to run an experiment which will depend on the user location? The existing architecture did not support such dynamic segments based on location, time zone etc.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*fWzUryq3kk4LJTpKKi7JSw.png" /></figure><p><strong>In order to get rid of these problems, create faster feature tests with experiments in production, and create a seamless rollout experience, we created an experimentation platform.</strong></p><h3>Meet Litmus 🖖</h3><p>Litmus is an experimentation platform (A/B or multivariate testing) for GOJEK.</p><p>It has two entities: <strong>Experiment</strong> and <strong>Release.</strong></p><p>Teams use <strong>Experiment </strong>when they want to experiment on a feature with multiple variants. For example, in the GOJEK app, we wanted to experiment with how many products to show on the homepage. That experiment looked something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bvP7zQJ74Ws9dRuX_kFqCA.png" /></figure><p>For any such feature as an experiment, there are three phases, ‘pre-experiment’, ‘during-experiment’, ‘post-experiment’.</p><p>The <strong>pre-experiment</strong> phase includes creating an experiment with data like No. of variants for that experiment, an initial value of traffic percent for the experiment and its variants.</p><p>The <strong>during-experiment</strong> phase includes testing with the different variants of feature and store data relevant to this experiment.</p><p>In <strong>post-experiment</strong> phase, experiment author will use the data collected during his experiment as feedback and decide on which variants of the experiment should be rolled out as a feature.</p><p>Teams use <strong>Release </strong>when they just want to release any feature gradually. One recent example where this was used was the gradual rollout of customer-to-customer chat to all our users.</p><p>Both the entities support rule and traffic percent. Teams can define rules like users in Jakarta <strong>and </strong>app version &gt;= 3.7 or user is power gofood user <strong>or</strong> power goride user for their experiment or release. The user will be part of the release/experiment only if they fit the rules.</p><p>Teams can also update the rules of their experiments or releases if they want to relax some or add more constraints. Traffic percent is used to control the experiment/release traffic. (It is defined in x% of the total GOJEK users).</p><p>Clients (any microservice/mobile app) make a request to Litmus with userID and some other information (which will be used in evaluating the rule) like app version , device os , location etc. The request for the user with some-user-id will be something like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2554d5adeda6e4968b9f1bd76cf327a4/href">https://medium.com/media/2554d5adeda6e4968b9f1bd76cf327a4/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*kGS7Ej8ejnOCxbmguWLVSw.png" /><figcaption>Fig 1: Litmus flow diagram</figcaption></figure><p>The response from Litmus will be along these lines:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0e32946b9962ed949cce2c3023a1eb7/href">https://medium.com/media/d0e32946b9962ed949cce2c3023a1eb7/href</a></iframe><p>The properties in the response will be used to decide what variation of a particular experiment needs to be shown to the user. In the case of the above example, the booking button will be shown at the bottom, and a discount card will be shown.</p><p>Now, back to those issues we mentioned earlier. Here’s how Litmus deals with them:</p><p>1. <strong><em>Adding and removing the users to the segments:</em></strong><em> </em>Feature rollout can be directly governed by the traffic percent and rule. When we want to increase the rollout, we can increase the traffic percent, no need to do any manual addition of users from a segment.</p><p>2. <strong><em>Roll out to 100% users:</em></strong><em> </em>Same as above, for rolling out a feature to 100%, we can just make the traffic percent value = 100. There is no need for fresh app release.</p><p>3. <strong><em>Dynamic Segments:</em></strong><em> </em>As explained above, the experiment/release supports rule . We can define a rule for a particular location or time for an experiment/release.</p><p>This was a very basic overview of how we designed Litmus and what it does. Watch this space for more updates about the cool things we do with it.</p><p>Thanks for reading!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*UMfKMvm9-j9lEu3mVUezmg.jpeg" /></figure><p>We’re always building interesting things at GOJEK. Some help our customers directly, others are geared towards our 2 million+ driver partners. Many others, like Litmus, aim to make life easier for our lean team of engineers. That said, we are expanding that team. If you want to join us and build interesting things, visit <a href="http://bit.ly/2RAeKGm">gojek.jobs</a> and apply. See you on the other side. 🙌</p><figure><a href="http://bit.ly/2RAeKGm"><img alt="" src="https://cdn-images-1.medium.com/max/892/1*PdKKvtpvxTQcbMlDvIlFzQ.png" /></a><figcaption>gojek.jobs</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3803467b6a53" width="1" height="1" alt=""><hr><p><a href="https://medium.com/gojekengineering/introducing-litmus-gojeks-own-experimentation-platform-3803467b6a53">Introducing Litmus: GOJEK’s Own Experimentation Platform</a> was originally published in <a href="https://medium.com/gojekengineering">Gojek Product + Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Grab a Slice on the Go]]></title>
            <link>https://medium.com/gojekengineering/grab-a-slice-on-the-go-c606344186c1?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c606344186c1</guid>
            <category><![CDATA[arrays]]></category>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Thu, 07 Feb 2019 03:56:55 GMT</pubDate>
            <atom:updated>2019-02-11T07:06:16.847Z</atom:updated>
            <content:encoded><![CDATA[<h4>How slices offer a dynamic, resizable alternative to arrays in Golang</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qPS_3O844KBN7xkBls6tzg.gif" /><figcaption><a href="http://bit.ly/2SthZSV">gojek.jobs</a></figcaption></figure><p>If you’ve used Golang, you might be familiar with the concept of arrays. In this post, we’re addressing slices. Why do slices exist when you already have arrays, you ask? We’ll talk about that, and a few more ideas, like:</p><ol><li>What are slices and why do we need them?</li><li>How do they work in Golang?</li><li>What mistakes are made using slices and how to avoid them.</li></ol><p>To understand the importance of slices, we need to talk arrays first. If you are familiar with arrays, then go ahead and jump directly to the section on slices.</p><h3>Arrays</h3><p>Similar to cpp, an array in Golang is a collection of the same type with continuous memory. You can access the elements directly through index.</p><p>Arrays in Golang are declared as [n]T where n is the size(length) of the array and T is the type, like int , string etc.</p><pre>var a [3]int</pre><pre>a[0] = 1</pre><pre>a[1] = 2</pre><pre>a[2] = 3</pre><pre>//or</pre><pre>a := [3]int{1, 2, 3}</pre><p>You can even ignore the length of the array in its declaration, and use ...instead of size ( n ).</p><pre>a := [...]{1, 2, 3}</pre><p>Here, the compiler will find the length for you.</p><p>Arrays in Golang are a value type, which means whenever you assign an array to a new variable, the copy of the original array is assigned to the new variable. This is also called deep copy .</p><pre>a := [...]string{&quot;Alice&quot;, &quot;Bob&quot;, &quot;Cop&quot;}</pre><pre>b := a // a copy of a is assigned to b</pre><pre>b[0] = &quot;Dytto&quot;</pre><pre>fmt.Println(&quot;a is &quot;, a)</pre><pre>fmt.Println(&quot;b is &quot;, b)</pre><p>The output of the above snippet will be:</p><pre>a is  [Alice Bob Cop]</pre><pre>b is  [Dytto Bob Cop]</pre><p>The main issue an array has is that it cannot be resized (which is a very generic requirement) as the length of the array is part of its type. For example, [3]intand [5]int are two different data types.</p><blockquote>This is where slices come in.</blockquote><h3>Slices</h3><p>Simply put, slices are the wrapper over arrays. Slices do not own any data of their own, they are just a reference to the existing array.</p><p>The syntax is like:</p><pre>a[start:end] creates a slice of array a from index start to end-1 .</pre><pre>a := [5]int{1, 2, 3, 4, 5}</pre><pre>var b []int = a[1:4] //creates a slice from a[1] to a[3]</pre><pre>a := []int{1, 2, 3} //creates a array of 3 integers and returns the slice reference which is stored in a.</pre><ul><li><strong>Why do we need slices?</strong></li></ul><p>Often, we have cases where we need to resize the length of the array (like append an array into another array). But arrays in Golang cannot be resized, hence we have slices, whose size can be dynamically changed.</p><p>Slices have length and capacity. <strong>Length</strong> is the number of elements present in the slice. <strong>Capacity</strong> is the number of elements present in the underlying array (to which the slice is referencing to), starting from the index from which the slice is created.</p><p>You can create a slice using make (like make([]T, len, cap) ) by passing length and capacity (capacity is an optional parameter and by default, it is equal to the length).</p><pre>a := make([]int, 5, 10) // a is a slice of length 5 and capacity 10.</pre><ul><li><strong>Modification of slices</strong></li></ul><p>As mentioned earlier, slices do not own any data on their own, they are just a reference to existing arrays. Hence, any modification done in the slice will reflect in the underlying array also. For example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/44d51a624621521d73c97fb1ee0f5d22/href">https://medium.com/media/44d51a624621521d73c97fb1ee0f5d22/href</a></iframe><p>The output will be:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d80a518db41c43e246fd24f21968a676/href">https://medium.com/media/d80a518db41c43e246fd24f21968a676/href</a></iframe><p>New elements can be added to the slice using append function. Like:</p><pre>readers := []string{&quot;Alice&quot;, &quot;Bob&quot;, &quot;Charlie&quot;}</pre><pre>fmt.Println(&quot;Readers&#39; array :&quot;, readers, &quot;has old length&quot;, len(readers), &quot;and capacity&quot;, cap(readers))</pre><pre>readers = append(readers, &quot;Dytto&quot;)</pre><pre>fmt.Println(&quot;Readers&#39; array :&quot;, readers, &quot;has new length&quot;, len(readers), &quot;and capacity&quot;, cap(readers))</pre><p>Here, the output will be:</p><pre>Readers&#39; array : [Alice Bob Charlie] has old length 3 and capacity 3<br>Readers&#39; array : [Alice Bob Charlie Dytto] has new length 4 and capacity 6</pre><p>Why has the capacity changed to 6? The answer is in the next section.</p><ul><li><strong>Memory allocation of slices</strong></li></ul><p>This is the most important aspect of slices. If you do not know how the size of slices can be dynamically changed, your code might not behave as you expected. Here’s an example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e6bbfa5ebeffcd0145756be81a1414b/href">https://medium.com/media/7e6bbfa5ebeffcd0145756be81a1414b/href</a></iframe><p>Most people would expect this output:</p><pre>a:  [1 2] <br>b:  [1 2 3] <br>c:  [1 2 4]<br>a:  [1 2 3] <br>x:  [1 2 3 4] <br>y:  [1 2 3 5]</pre><p>But the actual output looks like this:</p><pre>a:  [1 2] <br>b:  [1 2 3] <br>c:  [1 2 4]<br>a:  [1 2 3] <br>x:  [1 2 3 5] <br>y:  [1 2 3 5]</pre><p>The second one is correct. Let me explain.</p><blockquote>When the slice length is equal to its capacity and you try to append a new element to it, then the new memory is allocated to the slice which has double capacity as compared to the earlier capacity (most of the time).</blockquote><ul><li>So in case 1 at line 10, a has capacity and length both equal to 1,</li><li>In line 11, a gets allocated to a new address with capacity 2 (double the earlier one) and its length is also 2 (as two elements are present).</li><li>In line 12, a has capacity = length, hence line append(a, 3) creates a new array with capacity 4 (double the capacity of the earlier one) and returns the slice reference to b. Now b is a slice which has length = 3 and capacity =4. Here, the important point is that b is not referencing the underlying array of a , it is referencing a new array which has capacit = 4. a still has capacity and length =2.</li><li>In line 13, the same thing is happening as line 12, since a still has the length and capacity = 2, line append(a, 4) will create a new array with length 4 (double the capacity of a as a’s len(a) = cap(a)) and return the slice reference to c . Now c is also a slice with len = 3 and capacity =4. And same as line 12, c is not referencing to the underlying array of a , it is referencing to a new array which has capacity = 4. a still has capacity and length = 2.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*b229P1ODYGV34KsNHt8Jdw.png" /><figcaption>Case 1</figcaption></figure><ul><li>Before line 17, a’s length and capacity are both 2. But in line 17, a’s capacity has been changed as line a = append(a, 3) creates a new slice with capacity 4 (double a’s last capacity) and returns a slice reference to a itself. So after line 17, a has length = 3 (elements are 1, 2 and 3) and capacity =4.</li><li>Hence in line 18, x := append(a, 4) will append 4 to a (as a has capacity = 4 and currently has only 3 elements) and return the reference to x.</li><li>In line 19, same as line 18. y := append(a, 5) will append 5 to a (as ahas capacity = 4 and currently has 3 elements) and return the reference to y.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*taMU2CaEi3FmGOJhMe9quQ.png" /><figcaption>Case 2 (This is the scene after line 19)</figcaption></figure><p>The interesting fact is that in case 2, a, xand yare referencing the same array. Hence, changes done by y:= append(a, 5) are being reflected in xalso.</p><p>This is why the output is printing the same elements for x and y .</p><p>So, whenever you are working with slices and using append, beware of these scenarios.<strong> Also, one more important difference between array and slice is that you can compare two arrays but you cannot compare two slices in Golang. Why? 🤔 This is an exercise for the reader.</strong></p><blockquote>References:<br><a href="https://golangbot.com/arrays-and-slices/">https://golangbot.com/arrays-and-slices/</a></blockquote><blockquote><a href="https://blog.golang.org/go-slices-usage-and-internals">https://blog.golang.org/go-slices-usage-and-internals</a></blockquote><p>Clearly, we are blessed to have smart engineers at GOJEK, and we’re also scaling fast 📈. Our 18+ products are supported by 500+ microservices, so there’s no lack of fun problems to solve. If that sounds like your kind of work environment, check out <a href="http://bit.ly/2SthZSV">gojek.jobs</a>, and help us shake things up in Southeast Asia 🙌</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*PdKKvtpvxTQcbMlDvIlFzQ.png" /><figcaption><a href="http://bit.ly/2SthZSV">gojek.jobs</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c606344186c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/gojekengineering/grab-a-slice-on-the-go-c606344186c1">Grab a Slice on the Go</a> was originally published in <a href="https://medium.com/gojekengineering">Gojek Product + Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How slices internally work in golang]]></title>
            <link>https://riteeksrivastava.medium.com/how-slices-internally-work-in-golang-a47fcb5d42ce?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/a47fcb5d42ce</guid>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[pointers]]></category>
            <category><![CDATA[arrays]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Tue, 04 Dec 2018 19:42:07 GMT</pubDate>
            <atom:updated>2019-02-04T18:02:17.902Z</atom:updated>
            <content:encoded><![CDATA[<h3>How do slices internally work in golang</h3><p>Why do slices exists if you already have arrays in golang? Let’s see why we need slices in golang.</p><p>So in this blog, we will talk about</p><ol><li>What are slices and why do we need them?</li><li>How do they internally work in golang?</li><li>What mistakes can we make while using slices and how to avoid those mistake.</li></ol><p>For understanding the importance of slices, first, let&#39;s see what are arrays in golang? You are familiar with arrays then you can directly jump to slice section.</p><h3>Arrays</h3><p>Similar to cpp arrays in golang is the collection of the same type with continuous memory. (as you can access the elements directly through index).</p><p>Arrays in golang are declared as [n]T where n is the size(length) of the array and T is the type like int , string etc. Like:</p><pre>var a [3]int</pre><pre>a[0] = 1</pre><pre>a[1] = 2</pre><pre>a[3] = 3</pre><pre>//or</pre><pre>a := [3]int{1, 2, 3}</pre><p>You can even ignore the length of the array in the declaration of it. You can use ... instead of size ( n ), like</p><pre>a := [...]{1, 2, 3}</pre><p>Here the compiler will find the length for you.</p><p>Arrays in golang are a value type, means whenever you assign an array to a new variable then the copy of the original array is assigned to the new variable. This is also called as deep copy .</p><pre>a := [...]string{&quot;Alice&quot;, &quot;Bob&quot;, &quot;Cop&quot;}</pre><pre>b := a // a copy of a is assigned to b</pre><pre>b[0] = &quot;Dytto&quot;</pre><pre>fmt.Println(&quot;a is &quot;, a)</pre><pre>fmt.Println(&quot;b is &quot;, b)</pre><p>The output of the above snippet will be</p><pre>a is  [Alice Bob Cop]</pre><pre>b is  [Dytto Bob Cop]</pre><p>The main issue an array has is that it can not be resized(which is a very generic requirement) as the length of the array is part of it’s type i.e. [3]int and [5]int are two different data types.</p><p>To overcome this problem we have Slices in golang.</p><h3><strong>Slices</strong></h3><ul><li><strong><em>What are slices?</em></strong></li></ul><p>In very simple words slices are the wrapper over arrays. Slices do not own any data of their own, they are just a reference to the existing array.</p><p>The syntax is like:</p><pre>a[start:end] creates a slice of array a from index start to end-1 .</pre><pre>a := [5]int{1, 2, 3, 4, 5}</pre><pre>var b []int = a[1:4] //creates a slice from a[1] to a[3]</pre><pre>a := []int{1, 2, 3} //creates a array of 3 intergers and return the slice reference which is stored in a.</pre><ul><li><strong><em>Why do we need slices?</em></strong></li></ul><p>Very often we do have cases where we need to resize the length of the array. (like append an array into another array). But arrays in golang cannot be resized hence we have slices whose size can be dynamically changed.</p><p>Slices have length and capacity. <strong>Length</strong> is the number of elements present in the slice, <strong>capacity</strong> is the number of elements present in the underlying array(the array to which slice is referencing to) starting from the index from which the slice is created.</p><p>You can create a slice using make (like make([]T, len, cap) ) by passing length and capacity (capacity is an optional parameter and by default, it is equal to the length).</p><pre>a := make([]int, 5, 10) // a is a slice of length 5 and capacity 10.</pre><ul><li><strong><em>Modification of slices</em></strong></li></ul><p>As I mentioned earlier slices do not own any data on their own, they are just a reference to existing arrays. Hence any modification done in the slice will reflect in the underlying array also. For example,</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/44d51a624621521d73c97fb1ee0f5d22/href">https://medium.com/media/44d51a624621521d73c97fb1ee0f5d22/href</a></iframe><p>the output will be,</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d80a518db41c43e246fd24f21968a676/href">https://medium.com/media/d80a518db41c43e246fd24f21968a676/href</a></iframe><p>New elements can be added to the slice using append function. Like</p><pre>readers := []string{&quot;Alice&quot;, &quot;Bob&quot;, &quot;Charlie&quot;}</pre><pre>fmt.Println(&quot;Readers&#39; array :&quot;, readers, &quot;has old length&quot;, len(readers), &quot;and capacity&quot;, cap(readers))</pre><pre>readers = append(readers, &quot;Dytto&quot;)</pre><pre>fmt.Println(&quot;Readers&#39; array :&quot;, readers, &quot;has old length&quot;, len(readers), &quot;and capacity&quot;, cap(readers))</pre><p>the output will be</p><pre>Readers&#39; array : [Alice Bob Charlie] has old length 3 and capacity 3<br>Readers&#39; array : [Alice Bob Charlie Dytto] has new length 4and capacity 6</pre><p>Why the capacity changes to 6? The answer is in the next section.</p><ul><li><strong><em>Memory Allocation of slices</em></strong></li></ul><p>This is the most important part of slices. If you do not know how slices’ size can be dynamically changed then sometimes your code might not behave as you expected. Before the explanation of it lets see an example.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7e6bbfa5ebeffcd0145756be81a1414b/href">https://medium.com/media/7e6bbfa5ebeffcd0145756be81a1414b/href</a></iframe><p>most of the people will think that the output will be</p><pre>a:  [1 2] <br>b:  [1 2 3] <br>c:  [1 2 4]<br>a:  [1 2 3] <br>x:  [1 2 3 4] <br>y:  [1 2 3 5]</pre><p>But the output will be</p><pre>a:  [1 2] <br>b:  [1 2 3] <br>c:  [1 2 4]<br>a:  [1 2 3] <br>x:  [1 2 3 5] <br>y:  [1 2 3 5]</pre><p>And the later one is correct. I will explain it to you.</p><p><strong>When the slice length is equal to its capacity and you try to append a new element to it, then the new memory is allocated to the slice which has the double capacity as compared to the earlier capacity.</strong></p><ul><li>So in case 1 at line 10 a has capacity and length both equal to 1,</li><li>In line 11 a gets allocated to a new address with capacity 2 (double the earlier one) and its length is also 2. (as two elements are present).</li><li>In line 12 since a has capacity = length hence line append(a, 3) create a new array with length 4 (double the capacity of earlier one) and returns the slice reference to b. Now b is slice which has length = 3 and capacity =4. Here the important point is that b is not referencing to the underlying array of a , it is referencing to a new array which has length =4. a still has capacity and length =2.</li><li>In line 13 same thing is happening as line 12, since a still has the length and capacity = 2, line append(a, 4) will create a new array with capacity 4 (double the capacity of a as a’s len(a) = cap(a)) return the slice reference to c . Now c is also a slice with len = 3 and capacity =4. And same as line 13 c is not referencing to the underlying array of a , it is referencing to a new array which has length = 4. a still has capacity and length = 2.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*PiSPX0YMBjkd1fSnWlHURA.png" /><figcaption>Case 1</figcaption></figure><ul><li>Before line 17 a’s length and capacity both are 2. But in line 17 a’s capacity has been changed as line a = append(a, 3) creates a new slice with capacity 4 (double to the a’s last capacity) and return a slice reference to a only. So after line 17 a has length = 3 (elements are 1, 2 and 3) and capacity =4.</li><li>Hence in line 18 x := append(a, 4) will append 4 to a (as a has capacity = 4 and has currently only 3 elements) and return the reference (same as a referencing to) to x.</li><li>In line 19 same thing is happening as line 18. y := append(a, 5) will append 5 to a (as a has capacity = 4 and has currently 3 element) and return the reference (same as a referencing to ) to y.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*taMU2CaEi3FmGOJhMe9quQ.png" /><figcaption>Case 2 (This is the scene after line 19)</figcaption></figure><p>So interesting fact is that in case 2 a, xand yare referencing to the same array hence changes done by y:= append(a, 5) is being reflected in x also.</p><p>That is why the output is printing same elements for x and y .</p><p>So whenever you are working with slice and using append beware of such scenarios.</p><p>Reference:<br><a href="https://golangbot.com/arrays-and-slices/">https://golangbot.com/arrays-and-slices/</a></p><p><a href="https://blog.golang.org/go-slices-usage-and-internals">https://blog.golang.org/go-slices-usage-and-internals</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a47fcb5d42ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some concepts of Golang you should know.]]></title>
            <link>https://riteeksrivastava.medium.com/some-concepts-of-golang-you-should-know-a113ce572e38?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/a113ce572e38</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[goroutines]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[closure]]></category>
            <category><![CDATA[interview-questions]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Tue, 09 Oct 2018 21:10:26 GMT</pubDate>
            <atom:updated>2018-10-24T08:01:05.699Z</atom:updated>
            <content:encoded><![CDATA[<h3>Some concepts of Golang asked in Interviews</h3><h4><strong><em>1. Explain the go workspace. (</em></strong>src , bin<strong><em> , and </em></strong>pkg<strong><em> in GOPATH)</em></strong></h4><p>In go you have to first specify the location of your workspace before any development and GOPATH is an environment variable which specifies the location of workspace of your go projects. By default, it is assumed to be $HOME/go.<br>GOPATH has src , bin and pkg directories.</p><ul><li>src contains your go source files as packages.</li><li>bin contains executable commands like golint , gocov , glide , dep etc.</li><li>pkg contains package objects (suppose your project is in location $GOPATH/src/X/Y then the binary of the package with import path X/Y will be inpkg/{some-directory}/X/Y . {for more read <a href="https://github.com/golang/go/wiki/GOPATH">this</a> and <a href="https://thenewstack.io/understanding-golang-packages/">this</a>}</li></ul><h4><strong><em>2. What are goroutines and how are they different from threads?</em></strong></h4><p>For this to answer you should have a good knowledge of goroutines. In short, we can say <em>Goroutines are a way of doing tasks concurrently in golang. </em>For more about goroutines and their difference with threads please please read <a href="https://medium.com/@riteeksrivastava/a-complete-journey-with-goroutines-8472630c7f5c">this</a>. After the reading the mentioned article you will have enough content to answer the questions related to goroutines.</p><h4><strong><em>3. What are function closures? When should we use those? Can you give one example?</em></strong></h4><p><em>A closure is a function value that references variables from outside its body. The function may access and assign to the referenced variables; in this sense, the function is “bound” to the variables. (</em><a href="https://tour.golang.org/moretypes/25">reference</a><em>).</em></p><p>There are lots of use cases of closure. Like isolating data is one of them. Suppose if you want a function that has access to the data the persist even after the function exists. You can understand this with an example for implementation of Fibonacci series.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f3b2a466971ad48843d2badded733842/href">https://medium.com/media/f3b2a466971ad48843d2badded733842/href</a></iframe><p>In the above gist</p><pre>return func() int {<br>      n1, n2 = n2, (n2 + n1)<br>      return n1}</pre><p>this is called closure, and n1 and n2 are the variables that persist even after the execution of the function.</p><p>I have taken this example from <a href="https://www.calhoun.io/5-useful-ways-to-use-closures-in-go/">here</a>, you can visit this to see more example.</p><h4><strong><em>4. What are interfaces in GO?</em></strong></h4><p>In golang interface is a data structure which represents the signature of set of methods. &lt;Will add the detailed information&gt;</p><h4><strong><em>5. What is reflect? When should we use it?</em></strong></h4><p><a href="https://golangbot.com/reflection/">Reflection in Golang - golangbot.com</a></p><p>The above two links will just give you the overview of reflection in golang, and its pros and cons. For the internal implementation you should read about official blog of <a href="https://blog.golang.org/laws-of-reflection">reflection in go</a>.</p><h4><strong><em>6. How to copy a slice, map and an interface?</em></strong></h4><p>&lt;I am working on some examples, will add soon&gt;</p><h4><strong><em>7. What are channels in go? What kind of data-structure do they follow?</em></strong></h4><p>Please refer <a href="https://www.sohamkamani.com/blog/2017/08/24/golang-channels-explained/">this</a> for the best understanding of channels in golang.</p><p><a href="https://www.sohamkamani.com/blog/2017/08/24/golang-channels-explained/">An introduction to using and visualizing channels in Go ➡️</a></p><h4><strong><em>8. Does golang support pointer arithmetic?</em></strong></h4><p>No, golang does not support pointer arithemetic.</p><h4><strong><em>9. Does golang support inheritance?</em></strong></h4><p>No, golang does not support inheritance.</p><h4><strong><em>10. What is the difference between concurrency and parallelism?</em></strong></h4><p>Concurrency is dealing with multiple things at once (does not need to be done at the same time) with some time schedule while parallelism is doing the things at once at same time. For example if two threads are running on two different cores then it will be called parallelism but when two threads are scheduled on a single core with some scheduling algorithm then it will be called as concurrency.</p><p>If you want to add some more questions then please raise a PR at <a href="https://github.com/RiteekS/Golang-concepts">https://github.com/RiteekS/Golang-concepts</a>, I will add those here also with answers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a113ce572e38" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A complete journey with Goroutines]]></title>
            <link>https://riteeksrivastava.medium.com/a-complete-journey-with-goroutines-8472630c7f5c?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/8472630c7f5c</guid>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[goroutines]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Mon, 16 Apr 2018 05:32:22 GMT</pubDate>
            <atom:updated>2018-11-09T04:19:46.271Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xjHxBPHZVQP2uMfCT43XnA.jpeg" /></figure><p>When we want things to be done concurrently in Golang, we use Goroutines.</p><p>But wait. What is Concurrency?</p><p>Here’s an example: I’m writing this article and feel thirsty, I’ll stop typing and drink water. Then start typing again. Now, I’m dealing with two jobs (typing and drinking water) by some time slice, which is said to be concurrent jobs. Point to be noted here is that these two tasks (writing and drinking) are not being done at the same time. When things are being done at the same time it’s called parallelism.(Think checking your mobile AND eating chips).</p><p>So concurrency is dealing with multiple things at once (does not need to be done at the same time) with some time schedule and parallelism(doing multiple things at the same time) is a subset of this. If you are keen to know more about the difference between concurrency and parallelism, I highly recommend <a href="https://www.youtube.com/watch?v=cN_DpYBzKso&amp;t=422s">this</a> video.</p><h3>Overview</h3><p>In this blog, I will be talking about:</p><ul><li>What are Goroutines?</li><li>How are they different from threads?</li><li>Scheduling Goroutines</li><li>Common mistakes while using Goroutines and how to avoid them</li><li>Keywords</li></ul><h3>What are Goroutines?</h3><p>Goroutines are a way of doing tasks concurrently in golang. They allow us to create and run multiple methods or functions concurrently in the same address space inexpensively. The idea of Goroutines was inspired by <a href="https://en.wikipedia.org/wiki/Coroutine">Coroutines</a>. In my opinion, the only difference is Coroutines supports the explicit mean of transferring the control to other Coroutines while Goroutines have it implicitly (This point will get clearer in the scheduling section of this blog).</p><p>Goroutines are lightweight abstractions over threads because their creation and destruction are very cheap as compared to threads, and they are scheduled over OS threads. Executing the methods in the background is as easy as prepending the word go in a function call. Here’s a simple example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0587d746f1b8ef501c9cd5832af2e5b8/href">https://medium.com/media/0587d746f1b8ef501c9cd5832af2e5b8/href</a></iframe><p><a href="https://play.golang.org/p/UlpPlcIsyW1">Try</a></p><p>The output of the above program will be:</p><pre>My first goroutine<br>main function</pre><p>Remove time.sleepand run the program again. You will notice this output:</p><pre>main function</pre><p>To understand this we need to know how Goroutines are executed. In Golang, unlike normal functions, the control does not wait for the Goroutine to finish executing. The Control immediately returns to the next line of the code after a Goroutine call. The main Goroutine (main function is also a Goroutine) should be running for any other Goroutines to run. If the main Goroutine terminates, the program will be terminated and no other Goroutine will run.</p><p>Hence, when we are not using time.Sleep() then the controller immediately executes the line fmt.Println(“main function) just after the Goroutine call. After this, the main program gets terminated and we do not get the output of the Goroutine on the terminal.</p><p><em>Using sleep in the main Goroutine is a hack we are using just to understand how Goroutines work. Mainly, we use </em><a href="https://www.sohamkamani.com/blog/2017/08/24/golang-channels-explained/"><em>channels</em></a><em> to block the main Goroutine until all other Goroutines finish execution.</em></p><h3>How are they different from threads?</h3><p>Many think Goroutines are faster than threads. This is not entirely correct. It’s not any faster, but it allows you to do things concurrently. And if things are right, (like some processors are free) the runtime will also split the work across multiple processors. So you may get parallelism out of it, too. But think of it as a way of doing things concurrently. If task A is blocked on something (waiting on I/O), the scheduler understands that another Goroutine that’s ready to run can be executed while waiting for the I/O to return.</p><p>Goroutines have benefits over threads in: (inspired from <a href="https://blog.nindalf.com/posts/how-goroutines-work/">here</a>, I am just quoting)</p><p><strong>Memory consumption:</strong> The creation of Goroutines require much lesser memory as compared to threads. It requires 2kb of memory, while threads requires 1Mb(~500 times as compared to Goroutines). They are designed in a way that stack size of Goroutines can grow and shrink according to the need of an application. There might be only one thread in the program with thousands of Goroutines.</p><p><strong>Setup and Teardown cost:</strong> Threads have significant setups and teardown costs because it has to request resources from the OS and return it once it’s done. While Goroutines are created and destroyed by the go runtime (it manages scheduling, garbage collection, and the runtime environment for Goroutines) and those operations are pretty cheap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*ntxTfMNaxclAE7AJgBuAtw.png" /><figcaption>From: <a href="http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf">Analysis of Go Runtime scheduler</a></figcaption></figure><p><strong>Switch cost:</strong> This difference is mainly because of the difference in the scheduling of Goroutines and threads. Threads are scheduled <em>preemptively</em> (If a process is running for more than a scheduler time slice, it would preempt the process and schedule execution of another runnable process on the same CPU), the schedular needs to save/restore all registers i.e. 16 general purpose registers, PC (Program Counter), SP (Stack Pointer), segment registers etc.</p><p>While Goroutines are scheduled <em>cooperatively</em>, (explained in the scheduling section) they do not directly talk to the OS kernel. When a Goroutine switch occurs, very few registers (say 3) like program counter and stack pointer need to be saved/restored. For more details refer to <a href="https://electronics.stackexchange.com/questions/115286/what-processor-registers-are-saved-and-recovered-in-a-context-switch">this</a>.</p><h3>Scheduling of Goroutines</h3><p>As I have mentioned in the last paragraph, Goroutines are cooperatively scheduled. In cooperative scheduling there is no concept of scheduler time slice. In such scheduling Goroutines yield the control periodically when they are idle or logically blocked in order to run multiple Goroutines concurrently. The switch between Goroutines happen only at well defined points — when an explicit call is made to the Goruntime scheduler. And those well defined points are:</p><ul><li>Channels send and receive operations, if those operations would block.</li><li>The Go statement, although there is no guarantee that the new Goroutine will be scheduled immediately.</li><li>Blocking syscalls like file and network operations.</li><li>After being stopped for a garbage collection cycle.</li></ul><p>Now let’s see how they are scheduled internally. Go uses three entities to explain the Goroutine scheduling.</p><ul><li>Processor (P)</li><li>OSThread (M)</li><li>Goroutines (G)</li></ul><p>In any particular Go application the number of threads available for Goroutines to run is equal to the GOMAXPROCS, which by default is equal to the number of cores available for that application. We can use the runtime package to change this number at runtime as well. OSThreads are scheduled over processors and Goroutines are scheduled over OSThreads as explained in the Fig 2.</p><p>Golang has an M:N scheduler that also utilizes multiple processors. At any time, M goroutines need to be scheduled on N OS threads that runs on at most <a href="http://localhost:4000/computer%20science/2018/03/03/Complete-Journey-with-GoRoutines/#gomaxprocs">GOMAXPROCS</a>numbers of processors(N &lt;= GOMAXPROCS). Go scheduler distribute runnable goroutines over multiple worker OS threads that runs on one or more processors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bGkWfuAGbHK9r1lFpukYpw.png" /><figcaption>Inspired from <a href="https://rakyll.org/scheduler/">here</a></figcaption></figure><p>Every P has a local Goroutine queue. There’s also a global Goroutine queue which contains runnable Goroutines. Each M should be assigned to a P. Ps may have no Ms if they are blocked or in a system call. At any time, there are at most GOMAXPROCS number of P and only one M can run per P. More Ms can be created by the scheduler if required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q7_pAxBx1q4C8MUTHQ3oDg.png" /><figcaption>Inspired from <a href="https://rakyll.org/scheduler/">here</a></figcaption></figure><p>In each round of scheduling, scheduler finds a runnable G and executes it. Once a runnable G is found, it gets executed until it is blocked (as explained above).</p><pre>Search in the local queue<br>        if not found <br>            Try to steal from other Ps&#39; local queue //see Fig 3<br>            if not found <br>                Search in the global queue </pre><pre>        Also periodically it searches in the global queue (every ~ 1/70)</pre><h3>Common mistakes while using Goroutines and how to avoid them</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*elbQ6YZBF3x12ZfT9TRCXw.jpeg" /></figure><p><strong>Not checking the FD limit and memory limit:</strong> You should always check the resources (like file descriptor and memory) limit required for your Go program when you are creating a huge number of Groutines. Otherwise, if the memory limit will exceeds, then a memory leak may occur and if FD limit will exceed, then Goroutines may get stuck.</p><p><strong>Running Goroutines with infinite loops:</strong> Try executing the following golang code snippet.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a990242baff7c3cbcca3b710f987f2f/href">https://medium.com/media/9a990242baff7c3cbcca3b710f987f2f/href</a></iframe><p>Let’s execute the program:</p><pre>$ GOMAXPROCS=8 go run test.go</pre><p>We can see that the program never terminates. If we write the same program in C/C++, we will never observe such an issue. Now let’s rerun the program by changing the line: processors = runtime.GOMAXPROCS(0) -1. Now the program will terminate properly and prints the result. This is surprising isn’t it? you can read this <a href="http://www.sarathlakshman.com/2016/06/15/pitfall-of-golang-scheduler">blog</a> to read more about it.</p><h3>Keywords</h3><p><em>GOMAXPROCS</em></p><p>In the current version of Go, GOMAXPROCS is used to control the number of threads available for Goroutine execution to a particular go program. The hard limit is still on the number of CPU cores presented to the OS. The GOMAXPROCS option allows you to tune it down. By default, as of 1.5+ or 1.6+, GOMAXPROCS is set to runtime.NumCPU(). Fun trivia: in older versions of Go, it was set to 1, because the scheduler wasn’t as smart and GOMAXPROCS &gt; 1 was extremely detrimental to performance.</p><p><em>Any other hacks you can think of? Please do leave a comment.</em></p><p>And… We’re hiring. Check out <a href="http://gojek.jobs/">gojek.jobs </a>for more.</p><p>References</p><ul><li><a href="https://rakyll.org/scheduler/">https://rakyll.org/scheduler/</a></li><li><a href="https://blog.nindalf.com/posts/how-goroutines-work/">https://blog.nindalf.com/posts/how-goroutines-work/</a></li><li><a href="https://morsmachine.dk/go-scheduler">https://morsmachine.dk/go-scheduler</a></li><li><a href="http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf">http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8472630c7f5c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Complete Journey with Goroutines]]></title>
            <link>https://riteeksrivastava.medium.com/complete-journey-with-goroutines-f124298bd1d7?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f124298bd1d7</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[goroutines]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Mon, 02 Apr 2018 20:45:28 GMT</pubDate>
            <atom:updated>2018-05-03T13:43:02.346Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xjHxBPHZVQP2uMfCT43XnA.jpeg" /></figure><p>When we want things to be done concurrently in Golang, we use Goroutines.</p><p>But wait. What is Concurrency?</p><p>Here’s an example: I’m writing this article and feel thirsty, I’ll stop typing and drink water. Then start typing again. Now, I’m dealing with two jobs (typing and drinking water) by some time slice, which is said to be concurrent jobs. Point to be noted here is that these two tasks (writing and drinking) are not being done at the same time. When things are being done at the same time it’s called parallelism.(Think checking your mobile AND eating chips).</p><p>So concurrency is dealing with multiple things at once (does not need to be done at the same time) while some time schedule and parallelism is a subset of this. If you are keen to know more about the difference between concurrency and parallelism, I highly recommend <a href="https://www.youtube.com/watch?v=cN_DpYBzKso&amp;t=422s">this</a> video.</p><h3>Overview</h3><p>In this blog, I will be talking about:</p><ul><li>What are Goroutines?</li><li>How are they different from threads?</li><li>Scheduling Goroutines</li><li>Common mistakes while using Goroutines and how to avoid them</li><li>Keywords</li></ul><h3>What are Goroutines?</h3><p>Goroutines are a way of doing tasks concurrently in golang. They allow us to create and run multiple methods or functions concurrently in the same address space inexpensively. The idea of Goroutines was inspired by <a href="https://en.wikipedia.org/wiki/Coroutine">Coroutines</a>. In my opinion, the only difference is Coroutines supports the explicit mean of transferring the control to other Coroutines while Goroutines have it implicitly (This point will get clearer in the scheduling section of this blog).</p><p>Goroutines are lightweight abstractions over threads because their creation and destruction are very cheap as compared to threads, and they are scheduled over OS threads. Executing the methods in the background is as easy as prepending the word go in a function call. Here’s a simple example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0587d746f1b8ef501c9cd5832af2e5b8/href">https://medium.com/media/0587d746f1b8ef501c9cd5832af2e5b8/href</a></iframe><p><a href="https://play.golang.org/p/UlpPlcIsyW1">Try</a></p><p>The output of the above program will be:</p><pre>My first goroutine<br>main function</pre><p>Remove time.sleepand run the program again. You will notice this output:</p><pre>main function</pre><p>To understand this we need to know how Goroutines are executed. In Golang, unlike normal functions, the control does not wait for the Goroutine to finish executing. The Control immediately returns to the next line of the code after a Goroutine call. The main Goroutine (main function is also a Goroutine) should be running for any other Goroutines to run. If the main Goroutine terminates, the program will be terminated and no other Goroutine will run.</p><p>Hence, when we are not using time.Sleep() then the controller immediately executes the line fmt.Println(“main function) just after the Goroutine call. After this, the main program gets terminated and we do not get the output of the Goroutine on the terminal.</p><p><em>Using sleep in the main Goroutine is a hack we are using just to understand how Goroutines work. Mainly, we use </em><a href="https://www.sohamkamani.com/blog/2017/08/24/golang-channels-explained/"><em>channels</em></a><em> to block the main Goroutine until all other Goroutines finish execution.</em></p><h3>How are they different from threads?</h3><p>Many think Goroutines are faster than threads. This is not entirely correct. It’s not any faster, but it allows you to do things concurrently. And if things are right, (like some processors are free) the runtime will also split the work across multiple processors. So you may get parallelism out of it, too. But think of it as a way of doing things concurrently. If task A is blocked on something (waiting on I/O), the scheduler understands that another Goroutine that’s ready to run can be executed while waiting for the I/O to return.</p><p>Goroutines have benefits over threads in: (inspired from <a href="https://blog.nindalf.com/posts/how-goroutines-work/">here</a>, I am just quoting)</p><p><strong>Memory consumption:</strong> The creation of Goroutines require much lesser memory as compared to threads. It requires 2kb of memory, while threads requires 1Mb(~500 times as compared to Goroutines). They are designed in a way that stack size of Goroutines can grow and shrink according to the need of an application. There might be only one thread in the program with thousands of Goroutines.</p><p><strong>Setup and Teardown cost:</strong> Threads have significant setups and teardown costs because it has to request resources from the OS and return it once it’s done. While Goroutines are created and destroyed by the go runtime (it manages scheduling, garbage collection, and the runtime environment for Goroutines) and those operations are pretty cheap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/933/1*ntxTfMNaxclAE7AJgBuAtw.png" /><figcaption>From: <a href="http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf">Analysis of Go Runtime scheduler</a></figcaption></figure><p><strong>Switch cost:</strong> This difference is mainly because of the difference in the scheduling of Goroutines and threads. Threads are scheduled <em>preemptively</em> (If a process is running for more than a scheduler time slice, it would preempt the process and schedule execution of another runnable process on the same CPU), the schedular needs to save/restore all registers i.e. 16 general purpose registers, PC (Program Counter), SP (Stack Pointer), segment registers etc.</p><p>While Goroutines are scheduled <em>cooperatively</em>, (explained in the scheduling section) they do not directly talk to the OS kernel. When a Goroutine switch occurs, very few registers (say 3) like program counter and stack pointer need to be saved/restored. For more details refer to <a href="https://electronics.stackexchange.com/questions/115286/what-processor-registers-are-saved-and-recovered-in-a-context-switch">this</a>.</p><h3>Scheduling of Goroutines</h3><p>As I have mentioned in the last paragraph, Goroutines are cooperatively scheduled. In cooperative scheduling there is no concept of scheduler time slice. In such scheduling Goroutines yield the control periodically when they are idle or logically blocked in order to run multiple Goroutines concurrently. The switch between Goroutines happen only at well defined points - when an explicit call is made to the Goruntime scheduler. And those well defined points are:</p><ul><li>Channels send and receive operations, if those operations would block.</li><li>The Go statement, although there is no guarantee that the new Goroutine will be scheduled immediately.</li><li>Blocking syscalls like file and network operations.</li><li>After being stopped for a garbage collection cycle.</li></ul><p>Now let’s see how they are scheduled internally. Go uses three entities to explain the Goroutine scheduling.</p><ul><li>Processor (P)</li><li>OSThread (M)</li><li>Goroutines (G)</li></ul><p>In any particular Go application the number of threads available for Goroutines to run is equal to the GOMAXPROCS, which by default is equal to the number of cores available for that application. We can use the runtime package to change this number at runtime as well. OSThreads are scheduled over processors and Goroutines are scheduled over OSThreads as explained in the Fig 2.</p><p>Golang has an M:N scheduler that also utilizes multiple processors. At any time, M goroutines need to be scheduled on N OS threads that runs on at most <a href="http://localhost:4000/computer%20science/2018/03/03/Complete-Journey-with-GoRoutines/#gomaxprocs">GOMAXPROCS</a>numbers of processors(N &lt;= GOMAXPROCS). Go scheduler distribute runnable goroutines over multiple worker OS threads that runs on one or more processors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bGkWfuAGbHK9r1lFpukYpw.png" /><figcaption>Inspired from <a href="https://rakyll.org/scheduler/">here</a></figcaption></figure><p>Every P has a local Goroutine queue. There’s also a global Goroutine queue which contains runnable Goroutines. Each M should be assigned to a P. Ps may have no Ms if they are blocked or in a system call. At any time, there are at most GOMAXPROCS number of P and only one M can run per P. More Ms can be created by the scheduler if required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q7_pAxBx1q4C8MUTHQ3oDg.png" /><figcaption>Inspired from <a href="https://rakyll.org/scheduler/">here</a></figcaption></figure><p>In each round of scheduling, scheduler finds a runnable G and executes it. Once a runnable G is found, it gets executed until it is blocked (as explained above).</p><pre>Search in the local queue<br>        if not found <br>            Try to steal from other Ps&#39; local queue //see Fig 3<br>            if not found <br>                Search in the global queue <br><br>        Also periodically it searches in the global queue (every ~ 1/70)</pre><h3>Common mistakes while using Goroutines and how to avoid them</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/1*elbQ6YZBF3x12ZfT9TRCXw.jpeg" /></figure><p><strong>Not checking the FD limit and memory limit:</strong> You should always check the resources (like file descriptor and memory) limit required for your Go program when you are creating a huge number of Groutines. Otherwise, if the memory limit will exceeds, then a memory leak may occur and if FD limit will exceed, then Goroutines may get stuck.</p><p><strong>Running Goroutines with infinite loops:</strong> Try executing the following golang code snippet.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a990242baff7c3cbcca3b710f987f2f/href">https://medium.com/media/9a990242baff7c3cbcca3b710f987f2f/href</a></iframe><p>Let’s execute the program:</p><pre>$ GOMAXPROCS=8 go run test.go</pre><p>We can see that the program never terminates. If we write the same program in C/C++, we will never observe such an issue. Now let’s rerun the program by changing the line: processors = runtime.GOMAXPROCS(0) -1. Now the program will terminate properly and prints the result. This is surprising isn’t it? you can read this <a href="http://www.sarathlakshman.com/2016/06/15/pitfall-of-golang-scheduler">blog</a> to read more about it.</p><h3>Keywords</h3><p><em>GOMAXPROCS</em></p><p>In the current version of Go, GOMAXPROCS is used to control the number of threads available for Goroutine execution to a particular go program. The hard limit is still on the number of CPU cores presented to the OS. The GOMAXPROCS option allows you to tune it down. By default, as of 1.5+ or 1.6+, GOMAXPROCS is set to runtime.NumCPU(). Fun trivia: in older versions of Go, it was set to 1, because the scheduler wasn’t as smart and GOMAXPROCS &gt; 1 was extremely detrimental to performance.</p><p><em>Any other hacks you can think of? Please do leave a comment.</em></p><p>And… We’re hiring. Check out <a href="http://gojek.jobs/">gojek.jobs </a>for more.</p><p>References</p><ul><li><a href="https://rakyll.org/scheduler/">https://rakyll.org/scheduler/</a></li><li><a href="https://blog.nindalf.com/posts/how-goroutines-work/">https://blog.nindalf.com/posts/how-goroutines-work/</a></li><li><a href="https://morsmachine.dk/go-scheduler">https://morsmachine.dk/go-scheduler</a></li><li><a href="http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf">http://www.cs.columbia.edu/~aho/cs6998/reports/12-12-11_DeshpandeSponslerWeiss_GO.pdf</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f124298bd1d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understand Protocol Buffer basics]]></title>
            <link>https://riteeksrivastava.medium.com/protocol-buffer-what-is-it-41d1a3288e87?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/41d1a3288e87</guid>
            <category><![CDATA[protocol-buffers]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Sat, 23 Sep 2017 12:55:57 GMT</pubDate>
            <atom:updated>2018-03-30T11:39:25.675Z</atom:updated>
            <content:encoded><![CDATA[<p>In software development we often transfer data among servers or between servers and clients. And for the same we use different mechanisms and data formats. Protocol buffer is one of them. It is like JSON and XML but much faster than both. So here I will be talking about what it is, why is it faster than the others(JSON/XML) and why people are shifting towards it for most of the cases.</p><p>Initially proto buff was developed by google in early 2001 for internal server request/response protocol. Prior to this they were using marshalling and unmarshalling of request/response.(If you are curious to know about marshalling and unmarshalling please read <a href="https://howtodoinjava.com/jaxb/jaxb-exmaple-marshalling-and-unmarshalling-list-or-set-of-objects/">this</a>, otherwise it’s not very much relevant here). Later on 7 July 2008 it was made public.</p><p>Protocol buffer is a mechanism of serialising the data in faster and simpler way. It involves an<a href="http://whatis.techtarget.com/definition/IDL-interface-definition-language"> interface description language</a> to describe the structure of the data. We define our data structure or messages in .proto file and compile it with <a href="https://developers.google.com/protocol-buffers/docs/downloads">proto compiler</a> (of your preferred language).This compilation generates code that is invoked by sender or recipient. The compiled code will define classes for each message and service that .proto file defines. A schema for a particular use of protocol buffers associates data types with field names, using integers to identify each field. A .proto file looks something like this.</p><pre><strong>message</strong> <strong>Point</strong> {<br>  <strong>required</strong> int32 x = 1;<br>  <strong>required</strong> int32 y = 2;<br>  <strong>optional</strong> string label = 3;<br>}<br><br><strong>message</strong> <strong>Line</strong> {<br>  <strong>required</strong> Point start = 1;<br>  <strong>required</strong> Point end = 2;<br>  <strong>optional</strong> string label = 3;<br>}<br><br><strong>message</strong> <strong>Polyline</strong> {<br>  <strong>repeated</strong> Point point = 1;<br>  <strong>optional</strong> string label = 2;<br>}</pre><pre>// The above code follows the proto2 version</pre><p>In the above snippet “Point” message has two mandatory data items x and y and has a label which is optional. Similar to Point, Line and Polyline has data items and label. Both contain Point and demonstrate how composition works in Protocol Buffers. Here I have used three fields</p><ol><li><em>required : </em>used for mandatory data items,</li><li><em>options :</em> used for optional data item whose value may or may not present,</li><li><em>repeated : </em>it behaves like a vector.</li></ol><p>After this we run the protocol buffer compiler for application’s language on the above .proto file to generate data access classes. These provide simple accessors for each field (like x() and set_x()). After compilation of above .proto file three classes will be generated; Point, Line and Polyline. We can then use these classes in our application to populate, serialise, and retrieve protocol buffer messages. We can set the data like this(for cpp).</p><pre>Point point;<br>point.set_x(1);<br>point.set_y(0);<br>point.set_label(&quot;Unit length line from origin on x axis&quot;);<br>fstream output(&quot;myfile&quot;, ios::out | ios::binary);<br>point.SerializeToOstream(&amp;output);</pre><p>And later on we can read our message back in by using below code</p><pre>fstream input(&quot;myfile&quot;, ios::in | ios::binary);<br>Point point;<br>point.ParseFromIstream(&amp;input);<br>cout &lt;&lt; &quot;x_coordinate: &quot; &lt;&lt; point.x()&lt;&lt; endl;<br>cout &lt;&lt; &quot;y_coordinate:  &quot; &lt;&lt; point.y() &lt;&lt; endl;<br>cout &lt;&lt; &quot;Description: &quot; &lt;&lt; point.level() &lt;&lt; endl;</pre><p>We can add new fields to a message format without breaking backwards-compatibility, old binaries simply ignore the new field when parsing. This way it is backward and forward compatible. The flow of protocol buffer will look something like this</p><pre>        [Msg]                                         [Msg] <br>          |                                             |<br>          |                                             |<br>          |                                             |<br>          v                                             v<br>      [Encoding]-----------------------------------&gt;[Decoding]<br>           ^                                            ^<br>           |                                            |<br>           |                                            |<br>           |____________________________________________|<br>                                  |<br>                                  |<br>                                  | <br>                               [Proto]</pre><p>If you want to know about how encoding/decoding works then please see <a href="https://developers.google.com/protocol-buffers/docs/encoding">this</a>, it’s interesting.</p><p>Protocol buffer has two versions proto 2 and proto 3. You might be thinking that why they started with version 2, the reason is that the initial version of protocol buffers (“Proto1”) was developed in Google starting in early 2001 for the internal use so it was private.</p><p>It is much faster because it doesn’t send field names, rather the integers used to serialise those fields. On other side receiver also has the schema to decode the data received. One of the disadvantage I observed is that in case of protocol buffer sender and receiver both should have the knowledge of the schema. If you further want to know why protocol buffer is better than JSON or XML, simply google it, you will get a lot of links and answers. This is a good <a href="https://stackoverflow.com/questions/14028293/google-protocol-buffers-vs-json-vs-xml">link</a> for the comparison between JSON, XML and Protocol buffer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=41d1a3288e87" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hystrix: What is it and why is it used?]]></title>
            <link>https://riteeksrivastava.medium.com/hystrix-what-is-it-and-why-is-it-used-f84614c8df5e?source=rss-5b0a5456deaa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f84614c8df5e</guid>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Riteek Srivastav]]></dc:creator>
            <pubDate>Tue, 05 Sep 2017 18:37:01 GMT</pubDate>
            <atom:updated>2017-09-23T14:27:10.664Z</atom:updated>
            <content:encoded><![CDATA[<p>It is a library which is mainly used for the defence mechanism in the distributed environment and making system more resilient. I wanted to know the reason behind its name and came to know that Netflix wanted a name which would keep the theme of resilience, defence and tolerance mechanism while being short and not already taken, and they come with the name of this mechanism as Hystrix(To know more click <a href="https://github.com/Netflix/Hystrix/wiki/FAQ%20:%20General">here</a>).</p><p>In any distributed environment (with lot of dependencies), inevitably some of the many service dependencies fail. So Hystrix library is used to control the interaction between these distributed services by adding some latency tolerance and fault tolerance logic. It does so by the following methods:</p><ul><li>Isolating the points of access between the services</li><li>Stopping cascading failures in distributed system</li><li>Introducing timeout for particular action</li><li>Fallback and gracefully degrade when possible.</li></ul><p>Complex distributed architectures have lot of dependencies, any one of them will inevitably fail at some point. For example, an application having 50 services has 99.99% uptime for each of it’s services, thus we can expect that</p><p>over all uptime is 99.99⁵⁰ = 99.5%</p><p>0.5% of downtime</p><p>so 0.5% of 1 billion requests = 5,000,000 failures</p><p>this means 4hr+ downtime/month, even when all the dependencies have an excellent uptime individually.</p><p>We see that even when all dependencies perform well, the aggregate impact of even 0.01% downtime leads to 4 hour downtime per month if we do not make the whole system resilient. The above example or statistics shows the importance of Hystrix in any distributed system.</p><p>Hence, all of the failing and latency needs to be isolated and managed, so that a single failing dependency can not take down an entire application and whole of the system. Hystrix puts a wrapper around the dependencies and limits concurrent access to any one of them. If Hystrix library is used in the system, then the clients execute on separate threads. This isolates them from the calling threads, and so the caller may walk away from the dependency call that is taking a long time. When Hystrix is used then each dependencies is isolated form one other, restricted in the resources it can saturate when the latency occur and covered it in fall back logic that and covered in fallback logic that decides what response to make when any time of failure occurs in the dependencies. If you want to know the internal working and the flow chart of how it works then click <a href="https://github.com/Netflix/Hystrix/wiki/How-it-Works">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f84614c8df5e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>