<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Majid Fatemian on Medium]]></title>
        <description><![CDATA[Stories by Majid Fatemian on Medium]]></description>
        <link>https://medium.com/@majid?source=rss-234054763784------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*UUKZNqTD2NJ4iD4a4KIGkQ.jpeg</url>
            <title>Stories by Majid Fatemian on Medium</title>
            <link>https://medium.com/@majid?source=rss-234054763784------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:15:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@majid/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Instrumenting NewRelic for Databricks clusters]]></title>
            <link>https://medium.com/@majid/instrumenting-newrelic-for-databricks-clusters-e7ba87f7a42a?source=rss-234054763784------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7ba87f7a42a</guid>
            <dc:creator><![CDATA[Majid Fatemian]]></dc:creator>
            <pubDate>Sat, 23 May 2020 00:00:00 GMT</pubDate>
            <atom:updated>2020-05-26T02:46:02.310Z</atom:updated>
            <content:encoded><![CDATA[<p>In this quick tutorial we cover how to start using NewRelic’s amazing functionalities in our Apache Spark code that is running on Databricks clusters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TqcSwWfTfvyfsfCa.png" /></figure><p>Application monitoring systems are an inseparable part of software development, especially in distributed environments where applications like Hadoop and Apache Spark have become the defacto standard for processing large volumes of data efficiently.</p><p>If a system relies on third party services to get either of these, usually there is some amount of work involved to do the wiring and configuration to get them working together.</p><p>This article shows how to get the NewRelic’s Java agent up and running in Databricks pre-provisioned clusters or job clusters.</p><h3>Put the Agent on the Driver and Executors</h3><p>First step is to get the agent’s file downloaded or make it accessible on the servers. You need the NewRelic’s agent jar file and also the config file. The config file will be picked by the agent automatically.</p><p>We can manually upload these to a location in dbfs (Databrick’s File System) and be done with this, but if we have multiple clusters and you want slightly different configurations for each, you can do this through some <em>init scripts</em>.</p><p>Here is an example, which will create a <em>newrelic</em> subfolder inside the <em>/tmp</em>. Just keep in mind that the <em>init script</em> will be executed on the <em>Driver</em> as well as <em>All Executors</em>. So all of them will get an instance of the agent and the configuration, which is the result we are looking for.</p><p>For this example, we are just using the NewRelic’s default config template.</p><pre>#!/bin/bash </pre><pre># Creating the tmp and logs folder for NewRelic agent <br>mkdir -p /tmp/newrelic/logs <br>mkdir -p /tmp/newrelic/tmp </pre><pre>curl -o /tmp/newrelic/newrelic.jar https://download.newrelic.com/newrelic/java-agent/newrelic-agent/{{NEWRELIC_AGENT_VERSION}}/newrelic-agent-{{NEWRELIC_AGENT_VERSION}}.jar </pre><pre>curl -o /tmp/newrelic/newrelic.yml https://download.newrelic.com/newrelic/java-agent/newrelic-agent/{{NEWRELIC_AGENT_VERSION}}/newrelic.yml </pre><pre># Update the License Key as needed.</pre><p>Here is also where we can inject the <strong>NewRelic’s License Key</strong>. Retrieve it from wherever it is securely stored and replace it in the config file. Or we can just inject it as an environment variable which NewRelic’s agent will automatically picks it up.</p><pre>❗️Databricks File system is accessible at &quot;/dbfs&quot; on servers immediately as they come online. If we need to point to a resource in dbfs during an init script, it&#39;s preferred to use &quot;/dbfs&quot; to reference it rather than &quot;dbfs://&quot;. As it may have not been fully mounted while the init script is running.</pre><h3>Run Apache Spark with a Java Agent</h3><p>Next we need to tell the Driver and all Executors, to use a Java agent (NewRelic agent) along side the Apache Spark.</p><p>For this we need to pass that as an extra spark_conf under spark.driver.extraJavaOptions for the <em>Driver</em> and spark.executor.extraJavaOptions for the <em>Executors</em>.</p><p>But those configs are already set with some values by <em>Databricks</em>. To have our new values injected there, we basically need to find the existing ones, then append what we need on top of them, and pass the whole new setting to the cluster. Unfortunately there is no way to just append to the existing configs.</p><p>You can get the existing values from the <a href="https://docs.databricks.com/clusters/configure.html">cluster configuration page</a> or using <a href="https://docs.databricks.com/dev-tools/api/latest/jobs.html">jobs</a> / <a href="https://docs.databricks.com/dev-tools/api/latest/clusters.html">clusters APIs</a> (job clusters also have a cluster assigned to them). In your job or cluster configuration put those existing values in spark_conf section like below:</p><pre>&quot;spark_version&quot;: &quot;....&quot;, <br>&quot;spark_conf&quot;: { <br>    &quot;spark.driver.extraJavaOptions&quot;: &quot;...&quot;, <br>    &quot;spark.executor.extraJavaOptions&quot;: &quot;-Djava.io.tmpdir=/local/tmp -XX:ReservedCodeCacheSize=... -XX:+UseCodeCacheFlushing -Djava.security.properties=/databricks/spark/dbconf/java/extra.security&quot; <br>},</pre><p>then all you need is to add javaagent and newrelic.tempdir to those, pointing to the values that you have in your <em>init script</em> or where they are located in <em>dbfs</em>. Our final config would look like this:</p><pre>&quot;spark_version&quot;: &quot;....&quot;, <br>&quot;spark_conf&quot;: { <br>    &quot;spark.driver.extraJavaOptions&quot;: &quot;-javaagent:/tmp/newrelic/newrelic.jar -Dnewrelic.tempdir=/tmp/newrelic/tmp&quot;, <br>    &quot;spark.executor.extraJavaOptions&quot;: &quot;{{PREV_VALUES}} -javaagent:/newrelic/newrelic.jar -Dnewrelic.tempdir=/newrelic/tmp&quot; },</pre><p>Now the cluster is completely setup with NewRelic’s agent. As long as it finds the right <em>License key</em>, it can start communicating with NewRelic’s servers.</p><h3>Use NewRelic in code that runs on Databricks</h3><p>The final part of this, is to actually start using the agent in our code.</p><p>To add the dependencies, the pom.xml file would look something like below, similarly for the other package managers:</p><pre>&lt;dependency&gt; <br>    &lt;groupId&gt;com.newrelic.agent.java&lt;/groupId&gt;<br>    &lt;artifactId&gt;newrelic-agent&lt;/artifactId&gt;<br>    &lt;version&gt;{{NEWRELIC_AGENT_VERSION}}&lt;/version&gt;   <br>    &lt;scope&gt;provided&lt;/scope&gt; <br>&lt;/dependency&gt; <br>&lt;dependency&gt; <br>     &lt;groupId&gt;com.newrelic.agent.java&lt;/groupId&gt; <br>     &lt;artifactId&gt;newrelic-api&lt;/artifactId&gt; <br>     &lt;version&gt;{{NEWRELIC_AGENT_VERSION}}&lt;/version&gt; <br>     &lt;scope&gt;compile&lt;/scope&gt; <br>&lt;/dependency&gt;</pre><p>NewRelic’s agent will start picking up info from code automatically. If we need to specify which exact method to trace as <em>Transactions</em> or add <em>Custom Parameters</em> to it, we can use it like this in <em>Scala / Java</em>:</p><pre>import com.newrelic.api.agent.Trace <br>import com.newrelic.api.agent.NewRelic </pre><pre>... </pre><pre>// The @Trace annotation will create a new Transaction on NewRelic&#39;s side </pre><pre>@Trace(dispatcher = true) <br>def method_with_newrelic_tranx(): Unit = { </pre><pre>... </pre><pre>NewRelic.addCustomParameter(&quot;param&quot;, &quot;value&quot;) </pre><pre>... </pre><pre>} </pre><pre>...</pre><p>Congrats! Now we are all set.</p><p>We have created and installed NewRelic’s agent, have told our cluster to use it as a Java agent before spinning up Apache Spark Driver and Executors, and finally we have instrumented it in our code and started sending data to NewRelic.</p><p>Happy monitoring!</p><p>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a><a href="https://disqus.com/">comments powered by </a>Please enable JavaScript to view the</p><p><em>Originally published at </em><a href="https://majidfn.com/posts/20200523-databricks-newrelic-instrumentation/"><em>https://majidfn.com</em></a><em> on May 23, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7ba87f7a42a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Access denied to decrypt AWS Parameter Store]]></title>
            <link>https://medium.com/@majid/access-denied-to-decrypt-aws-parameter-store-635453978a3?source=rss-234054763784------2</link>
            <guid isPermaLink="false">https://medium.com/p/635453978a3</guid>
            <category><![CDATA[km]]></category>
            <category><![CDATA[application-development]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[encryption]]></category>
            <dc:creator><![CDATA[Majid Fatemian]]></dc:creator>
            <pubDate>Sun, 06 Oct 2019 16:59:25 GMT</pubDate>
            <atom:updated>2019-10-06T17:02:20.855Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Story was originally posted on <a href="https://majidfn.com/posts/20191005-paramstore-kms-access/">majidfn.com</a></blockquote><p>Use of <a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html">AWS parameter store</a> for storing the credentials, has made securing the applications much easier. You can encrypt the sensitive content like passwords, tokens and keys, then store them in parameter store and inject them to your application when they are needed.</p><p>Here is an example of the parameters stored there:</p><pre>aws ssm describe-parameters</pre><pre>{<br>    &quot;KeyId&quot;: &quot;alias/parameter_store_key&quot;, <br>    &quot;Name&quot;: &quot;/my-service/my-password&quot;, <br>    &quot;LastModifiedDate&quot;: 000.000, <br>    &quot;Version&quot;: 2, <br>    &quot;LastModifiedUser&quot;: &quot;---&quot;, <br>    &quot;Type&quot;: &quot;SecureString&quot;, <br>    &quot;Description&quot;: &quot;1&quot;<br>}, <br>{<br>    &quot;KeyId&quot;: &quot;alias/parameter_store_key&quot;, <br>    &quot;Name&quot;: &quot;/my-second-service/my-password&quot;, <br>    &quot;LastModifiedDate&quot;: 000, <br>    &quot;Version&quot;: 1, <br>    &quot;LastModifiedUser&quot;: &quot;---&quot;, <br>    &quot;Type&quot;: &quot;SecureString&quot;, <br>    &quot;Description&quot;: &quot;1&quot;<br>}</pre><p>And you can decrypt them easily like this:</p><pre>aws ssm get-parameter  --with-decryption --name /my-service/my-password</pre><pre>{<br>    &quot;Parameter&quot;: {<br>        &quot;Name&quot;: &quot;/my-service/my-password&quot;, <br>        &quot;LastModifiedDate&quot;: 000.000, <br>        &quot;Value&quot;: &quot;{{VALUE_REDACTED}}&quot;, <br>        &quot;Version&quot;: 2, <br>        &quot;Type&quot;: &quot;SecureString&quot;, <br>        &quot;ARN&quot;: &quot;arn:aws:ssm:us-east-1:{{ACCOUNT}}:parameter/my-service/my-password&quot;<br>    }<br>}</pre><p>Behind the scenes, Parameter store uses a KMS key to do the encryption and decryption of the content. In our example above the KMS key used for the encryption/decryption is alias/parameter_store_key. The service or user that wants to access those contents in parameter store, not only should have access to parameter store but also should have access to the KMS key that is used for encryption/decryption.</p><p>Following the least privilege principal, it’s much recommended to give the permission only to the single key that is used. So you can ensure that service / user can only access the sensitive content that it has permissions to and avoid accidents.</p><p>Here is an example of an IAM role that provides that:</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;VisualEditor0&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;ssm:DescribeParameters&quot;,<br>                &quot;ssm:GetParametersByPath&quot;,<br>                &quot;ssm:GetParameters&quot;,<br>                &quot;ssm:GetParameter&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        },<br>        {<br>            &quot;Sid&quot;: &quot;VisualEditor1&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: &quot;kms:Decrypt&quot;,<br>            &quot;Resource&quot;: [<br>                &quot;arn:aws:kms:us-east-1:{{ACCOUNT}}:key/ef1d677e-92ac-4710-9124-52f2844f15ee&quot;<br>            ]<br>        }<br>    ]<br>}</pre><p>As you can see, Parameter store is using the KMS key’s alias, alias/parameter_store_key, but the permission is defined based on the arn, arn:aws:kms:us-east-1:{{ACCOUNT}}:key/ef1d677e-92ac-4710-9124-52f2844f15ee.</p><h3>AccessDeniedException when calling the GetParameter operation</h3><p>I encountered an interesting issue in the past couple of days which I thought worths sharing.</p><p>Both parameters in the parameter store were using the same KMS key alias to encrypt and decrypt, and the first one was decrypting fine, but the second one was raising:</p><pre>aws ssm get-parameter  --with-decryption --name /my-second-service/my-password</pre><pre>An error occurred (AccessDeniedException) when calling the GetParameter operation: <br>The ciphertext refers to a customer master key that does not exist, <br>does not exist in this region, or you are not allowed to access. <br>(Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException;)</pre><p>when I changed the permission in the IAM role to key/* and basically allowing access to all the available keys, the exception was resolved and I could get the decrypted value of the encrypted content, as if it was using another key behind the scenes.</p><p>But again, that was not desired as we want to go with the least privilege access route.</p><p>It was becoming a mystery why two params, using the exact same key, one could be decrypted and one couldn’t.</p><p>To solve the mystery we referred to our good friend CloudTrail. To understand what is actually going on, and why one key could be decrypted and one not. As always our friend, CloudTrail, didn’t let us down.</p><p>By investigating further we realized that during our initial setup, we had created a KMS key first, assigned it the same alias alias/parameter_store_key, then created the parameter store entries. According to CloudTrail, the same key alias was later on deleted and then assigned to <em>another KMS key</em>. So although the parameter store shows the same KMS key alias for both params, but behind the scenes they were referring to two different keys. Which unfortunately there is no way to get the key ARN used by parameter store, it only shows the alias.</p><p>There we go, our theory that the second parameter needed access to a another KMS key was correct.</p><p>A simple resolution and safer to this is to rewrite the values in parameter store. When rewritten, it will use the latest KMS key, and for all the users of that entry in parameter store we can give them access to a single key.</p><blockquote>Although KeyId aliases for both parameter store values are the same, but they might point to different KMS keys behind the scenes. This happens if the alias was deleted and re-assigned after the parameter store values got created.</blockquote><p>On a side note, it is recommend to use Segment’s <a href="https://github.com/segmentio/chamber">Chamber</a> which is a wrapper around Parameter Store and makes interactions with it easier. It automatically injects the values as <em>ENV VARs</em> when needed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=635453978a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Securing MicroServices]]></title>
            <link>https://medium.com/rv-data/securing-microservices-d029ec4a9662?source=rss-234054763784------2</link>
            <guid isPermaLink="false">https://medium.com/p/d029ec4a9662</guid>
            <category><![CDATA[authentication]]></category>
            <category><![CDATA[authorization]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Majid Fatemian]]></dc:creator>
            <pubDate>Thu, 06 Sep 2018 20:15:14 GMT</pubDate>
            <atom:updated>2018-09-06T21:28:20.656Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mjihLfdTzn9YInh3TOeaGA.jpeg" /></figure><p>At RedVentures we have been benefiting from migrating our architectures to MicroServices for a few years. These fine grained services have helped make us technology and protocol agnostic with resilience and elasticity.<br>Apart from all the benefits (and challenges), one aspect remains really important as we grow in number of different services: <strong>security</strong>.</p><p>It is extremely critical for us to control who has access to what and who is allowed to perform actions.</p><h3>Authentication vs. Authorization</h3><p>When we are talking about security, it’s important to distinguish between these two:</p><ul><li><strong>Authentication</strong> is verifying the identity of a user or process.</li><li><strong>Authorization</strong> is permitting performing of a specific action to a user or a process.</li></ul><h3>Authentication</h3><p>Identifying a user or process is a very critical operation that involves checking credentials and other sensitive data. A simple mistake could result in disasters. Luckily, there are numerous solutions to solve this problem and our approach for such sensitive matter is: <strong><em>reuse instead of build.</em></strong><em> <br></em>Encouraging to reuse what’s already built by experts. Our Authentication solution consists of the followings:</p><p><strong>1 — Active Directory<br></strong>Provides simple and easy to use authentication for our internal users. But we have encountered other scenarios, like internal users want to use external SaaS applications and Cloud providers. Also our external partners are willing to have access to some of the internal tools provided for them. Therefore we have to extend out Authentication using:</p><p><strong>2 — Okta<br></strong>Okta syncs with our ActiveDirectory system and provides access for our internal and external users by providing <a href="https://en.wikipedia.org/wiki/Security_Assertion_Markup_Language">SAML</a> (Security Assertion Markup Language). The SAML is signed using a private key by the identity provider (Okta) and each application has access to the corresponding public key, so they can assure if the SAML is legitimately signed or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3y2HCCMnQjQ7OTVNM3xAfQ.jpeg" /></figure><p>SAML is XML based and we have long passed dealing with XML and its human unfriendliness, that’s why we introduced:</p><p><strong>3 — JWT <br></strong><a href="https://en.wikipedia.org/wiki/JSON_Web_Token">JSON Web Token</a> has made authentication a lot easier by providing an access token which asserts some claims in it. The same signing/verification process as above ensures that the token and the claims are legitimate, and they can be used by different services to verify the validity of the user or process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PpLqVzaLQgn1SUz7dLMwRQ.jpeg" /></figure><p>The structure of JWT is base64 encoded values of following three sections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_SK4lDpsTkfyAeSCGZFA3A.jpeg" /></figure><p>Header provides some meta data about the token. Signature ensures the validity of the identity provider and the payload itself. Payload after being decoded gives a JSON object similar to the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L-lLqq4fIsqp_Zw8xXtM8g.jpeg" /></figure><p>Keep in mind that no sensitive data should be included in the JWT, all the data is easily decodable.</p><h3>Authorization</h3><p>After ensuring that each MicroService is aware of the identity of the user or process accessing it, we need to ensure that the action is permitted to be performed on that service by the user. We have multiple partners which may use the same MicroService but we have to ensure that they have only access to the parts and actions they are allowed and nothing more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u2BmAHYPlRpGFBdGDJVO0Q.jpeg" /></figure><p><strong>The problem</strong><br>Historically, each of our services implemented Authorization differently. Some were using ActiveDirectory groups to permit/block users and actions. Some were having their own custom implementations. It was scattered, inconsistent and very difficult to monitor.</p><p><strong>The desired Solution</strong><br>We wanted our Authorization to have the following criteria to make it adaptable company-wide, as well as improving our security:</p><ul><li>Decentralized</li><li>Simple</li><li>Scalable</li><li>Monitoring</li><li>Centralized Management</li></ul><p>The majority of existing solutions would make a round trip to a central authorization server for each request to verify permissions. Due to the large number of round trips, we experienced significant network traffic and latency in our applications. We knew this could not scale and needed a self-contained service to avoid any round trips.<br>Besides that, many of the existing solutions support <em>Roles</em>, <em>Groups</em> and <em>Access Levels </em>and having flexibility of fine grained management and control comes with a price of additional complexity.</p><p>Unnecessary complexity makes the application prone to errors and therefore vulnerabilities.</p><p><strong>Simplified Authorization<br></strong>As mentioned earlier, each MicroService could serve different business partner so access and performed actions on each should be controlled.<br>We have defined this level of control within each MicroSearch as an <strong><em>Area</em></strong>. A user or a process that has access to all the <em>Areas</em> within the MicroService has a <strong><em>Global</em></strong> access.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wJogAwFuufezS7i5Og5uDw.jpeg" /></figure><p>Looking at the organization of these access controls we can easily fit them in to a JSON structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2hmNCZ70C6p-4GbVmt8T8Q.jpeg" /></figure><p>JWT goes hand in hand with JSON, therefore we embedded the Authorization data inside the JWT.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2wN_cmNS3Yog79m0X7qlsQ.jpeg" /></figure><p>Having Access Control data inside JWT provides a lot of flexibility. After logging in, each user gets assigned a JWT which includes Access control data. Each MicroService is not only able to Authenticate the user, but also to Authorize the user as well and allow/block certain actions based on the permissions.<br>This validation happens as a self-contained process, there is no need for a central service to verify an access therefore there is no round-trip.</p><p><strong>The Big Picture</strong><br>Overall the architecture looks like the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bsBYrTfnJVvZ9prwVw7KGA.jpeg" /></figure><p>After logging in with Okta, user gets redirected to a JWT identity provider, before generating the token, providers communicates with an Authorization service and gets all the permissions for the user, merges it with other data and then generate the token and send it back to the user.</p><p><strong>Ease of integration by providing SDKs</strong><br>With the current integration as is, we have checked 3 out of 5 checkboxes that we defined above. It is a <strong><em>decentralized</em></strong> system which is <strong><em>simple</em></strong> and <strong><em>scalable</em></strong>. <br>To make integration easier we provided the SDKs for major languages being used in our technology stacks, including C#, Go, NodeJS and PHP.</p><p><strong>Monitoring</strong><br>In the SDKs we have embedded the Monitoring on who is accessing which resource within which Area and what’s the action they are taking. By shipping all the logs to ElasticSearch we could also put alerts on top of suspicious attempts to draw attention to dig in to the activities of that particular user or process.</p><h3>Conclusion</h3><p><strong>Reuse &gt; Recreate</strong><br>Security is a sensitive topic and very important. But there are plenty of tools already built which have been proven over time and they are actively maintained and patched. Try to reuse what is already built and proven rather than recreating it.</p><p><strong>Token based Auth<br></strong>Token based Auth like JWT is scalable, decentralized, stateless and easy to use. Also you can embed custom data in to parts of it which could be retrieved and used in any application. Keep in mind that no sensitive data should be embedded in the token based authentication as it is decodable easily. Signature is only to verify the validity of data.</p><p><strong>Simple does not mean Insecure<br></strong>When it comes to security try to keep things simple. Security does not necessarily need to be complex. Over complicating the case might work counter productive and make the setup prone to vulnerabilities. Keeping security simple makes it easier to monitor, trace and debug.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d029ec4a9662" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rv-data/securing-microservices-d029ec4a9662">Securing MicroServices</a> was originally published in <a href="https://medium.com/rv-data">Red Ventures Data Science &amp; Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Business value of Polyglot programming]]></title>
            <link>https://medium.com/rv-data/business-value-of-polyglot-programming-de0942dd1833?source=rss-234054763784------2</link>
            <guid isPermaLink="false">https://medium.com/p/de0942dd1833</guid>
            <category><![CDATA[polyglot]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Majid Fatemian]]></dc:creator>
            <pubDate>Wed, 06 Sep 2017 20:58:22 GMT</pubDate>
            <atom:updated>2017-09-08T14:15:21.328Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*ZSM8nSVlPYOtPwGfaZZGLw.png" /></figure><p>Many years ago, after working with PHP for about 8 years, I was pushed to start development in Python for the first time. My mentality at the time was to focus on one thing and become the expert on the subject. I wanted to be the expert of PHP &amp; MySQL world and thought moving away from it would make me less focused on the subject.</p><p>I was very resistant to this change and facing a new world.</p><p>As I delved more into Python and learning it, I realized I was solving almost the same problems but in a very different approach. Features of the language are very similar, but the way they are used is different. The community of Python developers approach problems in a different way than PHP developers. I started learning their approach and how they thought and approached problems. And started liking it.</p><p>As I became more comfortable with Python, I started thinking through problems differently. Since I knew two languages, I had the option of approaching problems in two different ways. Whichever best solved the problem, won.</p><p>A few years passed, and again I switched projects. And again, I learned another language: This time, Node.JS. The language was not new to me, but the stack was. But just as I had lost my prejudice on programming languages, I was willing to take the opportunity to learn the platform and polish my skills.</p><p>I learned that in practice that each language has its own advantages and each is good to solve particular problems. The more I learned different technology stacks, the less I became biased in solving problems with a certain technology stack or language. When thinking about solving a problem, I spend most of my time thinking what the problem actually is, the best approach to solve it, and what tool fits the best for that approach. I then pick the best tool for the problem out of the toolbox.</p><p>Being technology and language agnostic helps the development teams to approach the problem with the best possible tools but also introduces its own business values and challenges. When you use the best tool for the job, development time decreases, the code base simplifies, and you stand to see an increase the overall performance.</p><p>However, as with any other technological and architectural choices, there are advantages and disadvantages associated with Polyglot technologies.</p><h3>Simply solving the problem</h3><p>When choosing multiple technologies to co-exist in an environment, the goal is to pick the best solution for the problem in hand. By picking the appropriate technology the implementation become easier and simpler.</p><p>So if you are building a bi-directional notification system, you can choose the technology stacks that are most suited for this.<br> Or if you want to have high performance data throughput, there are variety of different technologies to look at.</p><p>Of course a single language is capable of doing a lot, but when you pick the stack that’s more appropriate for a particular problem, you will have less time spent on trying to bend the problem to fit with the capabilities of the chosen language.</p><h3>Performance</h3><p>No single language is best at solving all problems. When it is needed to solve a problem with a specific technology which is not designed for that, it takes more time from the development team to handle it. Also it is very probable that the actual performance is affected by numerous tweaks that developers had to do. Also usually more work is involved to adapt a technology to do something which is not initially designed for. By picking the right technology stack, not only it reduces the development time, but also the overall performance of that component is highly affected as well.</p><h3>You have a toolbox, not only hammer</h3><p>When you have variety of tools from which to choose, you will pick the tool that is best for the problem, not the one with which you are most comfortable.</p><p>Abraham Maslow said, “I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.” When the development team has more tools in their toolbox, only nails look like nails.</p><h3>Team of full-stack developers</h3><p>I believe architecture of a software application is a collaborative responsibility. Everyone in the team should have a firm understanding of the architecture.</p><p>The architecture includes the full spectrum of the software development, build, deployment, and operations.<br> Everyone in the team should understand the big picture of what is happening under the hood. This helps the entire team to have inputs for the architecture of the application and everyone is equally responsible. Also in cases of emergencies, every individual within the team is capable of jumping in and help.</p><h3>DevOps and Maintenance</h3><p>Operation of Polyglot systems is the biggest concern for development team.</p><p>When the number of different technologies grows, it becomes more difficult for the DevOps team to maintain the application and infrastructure. Each language and technology requires its specific environment, setup, and skill set to monitor and maintain.</p><p>Complexity of a system grows exponentially. The more complex a system gets the more difficult it becomes for maintenance.</p><h3>No better time than now</h3><p>Being able to use polyglot technologies and programming languages is not a new idea. Throughout the history of software development there have been many attempts to facilitate this and make integration of completely different technologies easier. CORBA, Java RMI, and different RPC methods are all examples of different approaches to solve the same problem; acting as a liaison to make different technologies communicate with one another.</p><p>It has never been easy to do so. Each of these technologies has its own drawbacks, which made them popular for certain situations. But with the increasing popularity and growth of containers, it is easier to package an application with its dependencies into one isolated image, which in turn makes it easy to deploy anywhere. Containers, in combination with microservices, makes polyglot technologies a good solution.</p><p>In a microservices approach, an application is broken into several smaller services, each could potentially be built on a different technology stack. But when packaged in a container it is ready for deployment in any environment, a lot easier.</p><p>The beauty of microservices is the isolation of each component. You can assign a single responsibility to each component and use a technology stack that best solves that particular responsibility. Each component provides a standard interface with which the rest of the microservices communicate.</p><h3>Conclusion</h3><p>When facing a technical problem, being technology agnostic enables you to pick the best tool for the job. You can also break a problem in to several smaller components and pick the best suitable technology for each.</p><p>With the emerge of containers and microservices, it is more convenient than ever to build software systems consisting of several different technologies and have them communicate with one another. One main concern is complexity. The more technologies we have, the more complex it becomes to monitor and maintain a system.</p><p>Software engineering is the science of knowing and understanding the tools and picking the right ones for the job.</p><p>RV Tech has been a great example of this. In the past few years, we have been able to decouple our monolith applications and break them in to tiny microservices and use variety of different technologies to solve our problems.<br> The beauty lies under the fact that we don’t necessarily need to be certain for a specific technology when we are picking it. It’s small enough so it can be easily replaced. Relatively!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=de0942dd1833" width="1" height="1" alt=""><hr><p><a href="https://medium.com/rv-data/business-value-of-polyglot-programming-de0942dd1833">Business value of Polyglot programming</a> was originally published in <a href="https://medium.com/rv-data">Red Ventures Data Science &amp; Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>