<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mert Çalışkan on Medium]]></title>
        <description><![CDATA[Stories by Mert Çalışkan on Medium]]></description>
        <link>https://medium.com/@mertcal?source=rss-56c3e2f1314c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*D0XlG1TQ9cIzvOuk.jpeg</url>
            <title>Stories by Mert Çalışkan on Medium</title>
            <link>https://medium.com/@mertcal?source=rss-56c3e2f1314c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 09:25:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mertcal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Executing Cloudformation bash scripts with sudo]]></title>
            <link>https://medium.com/@mertcal/executing-cloudformation-bash-scripts-with-sudo-f23d5f14ae07?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/f23d5f14ae07</guid>
            <category><![CDATA[amazon]]></category>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[sudo]]></category>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Mon, 02 Jan 2017 12:30:10 GMT</pubDate>
            <atom:updated>2017-01-02T12:30:10.274Z</atom:updated>
            <content:encoded><![CDATA[<p>The biggest catch for Cloudformation is to execute your bash scripts (a.k.a UserData) after spawning your instance but in order to execute them with sudo you need to execute first:</p><pre><strong>sed -i &#39;s/Defaults    requiretty/Defaults    !requiretty/g&#39; /etc/sudoers</strong></pre><p>and then rollback with:</p><pre><strong>sed -i &#39;s/defaults    !requiretty/defaults    requiretty/g&#39; /etc/sudoers</strong></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f23d5f14ae07" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to silently change admin password and enable secure admin on Payara]]></title>
            <link>https://medium.com/@mertcal/how-to-silently-change-admin-password-and-enable-secure-admin-on-payara-357a6d3dce04?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/357a6d3dce04</guid>
            <category><![CDATA[payara]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Sun, 01 Jan 2017 21:54:02 GMT</pubDate>
            <atom:updated>2017-01-03T11:02:06.862Z</atom:updated>
            <content:encoded><![CDATA[<p>While installing Payara on Amazon EC2 instances, I wanted to enable accessing the admin console remotely. This is easy to handle with <strong>change-admin-password </strong>and <strong>enable-secure-admin</strong> asadmin commands but since I am using Cloudformation for shaping up my EC2 cluster and installing Payara onto them, asadmin commands that prompt for user information and password is no good. It’s because I’m heavily dependant on Cloudformation’s bash script execution mechanism.</p><p>For executing <strong>change-admin-password </strong>and <strong>enable-secure-admin</strong> asadmin commands you need to provide user and <em>passwordfile</em> attributes to those commands as:</p><pre><strong>asadmin --user admin --passwordfile /path/to/old_passwordfile.txt change-admin-password</strong></pre><pre><strong>asadmin --user admin --passwordfile /path/to/new_passwordfile.txt enable-secure-admin</strong></pre><p>The content of the old_passwordfile.txt could be like:</p><pre>AS_ADMIN_PASSWORD=<br>AS_ADMIN_NEWPASSWORD=MyPassyRocks*</pre><p>where I state that asadmin password is empty and the new one will be <strong>MyPassyRocks*</strong>. And the content of the new_passwordfile.txt could be like:</p><pre>AS_ADMIN_PASSWORD=MyPassyRocks*</pre><p>where I state the changed asadmin password. With this configuration it’s possible to change the asadmin password without any prompting for user info and passwords.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=357a6d3dce04" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Cassandra Cluster with Kitematic]]></title>
            <link>https://medium.com/@mertcal/creating-cassandra-cluster-with-kitematic-88fe71c2976f?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/88fe71c2976f</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Wed, 16 Sep 2015 09:40:55 GMT</pubDate>
            <atom:updated>2015-09-16T09:40:55.990Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.docker.com/toolbox">Docker Toolbox</a> is a nice toolset from Docker that offers:</p><ol><li>Docker Client</li><li>Docker Machine</li><li>Docker Compose (Mac only)</li><li><a href="http://kitematic.com/">Docker Kitematic</a></li><li>VirtualBox</li></ol><p>I’ve been using the Kitematic since its early stages and it eases the configuration &amp; setup of docker containers a lot with its GUI. After seeing Cassandra released an official image, I wanted to to give it a try by using Kitematic.</p><p>When you open up the Kitematic and search for the cassandra, you will end up with a result list as given follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pX5MccuUwK7CjhJqFvscNw.png" /></figure><p>So by clicking the Create on the official cassandra image will result to downloading the image itself and then creating a container out of it as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GdBK7RdG_6jGMl9oVINJ6A.png" /></figure><p>After creating my first cassandra container I wanted to rename it to cassandra-1 since 2 more will be on its way :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4G6BZYLqs3nwIblyU_Ot8A.png" /></figure><p>For getting 2 more containers up and running, I opened up the Docker CLI (click on the left bottom button named as that). So that opens up a bash of my virtual machine. Now I’m ready to execute some docker commands to create 2 more containers. Commands are given as follows. They need to be executed in order. Please follow on with the post to know how to execute them properly.</p><pre>docker run — name cassandra-2 -d -e CASSANDRA_SEEDS=&quot;$(docker inspect — format=’{{ .NetworkSettings.IPAddress }}’ cassandra-1)&quot; cassandra:latest</pre><pre>docker run — name cassandra-3 -d -e CASSANDRA_SEEDS=&quot;$(docker inspect — format=’{{ .NetworkSettings.IPAddress }}’ cassandra-1)&quot; cassandra:latest</pre><blockquote><strong>Keep in mind that copy paste of code snippets might arise problems on “ and ”. Do replace them after copying.</strong></blockquote><p>So the commands given create containers by setting the CASSANDRA_SEEDS environment variable with the IP value of the first container. The IP is resolved by another inline docker command.</p><p>After executing the first command for creating the <strong>cassandra-2</strong> container, I moved back to kitematic and opened up terminal for cassandra-1 container by clicking Exec button pointed in the screenshot as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DA50Oe40FeFWlSEr3JdV4g.png" /></figure><p>This opens up a terminal for the 1st container. I executed the nodetool status command to check the status of my 2nd container. When you see the UN (Up Normal) for both of the containers as follows, you can continue with creating the 3rd container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sV81RsANZehWNU2wx1I2Ug.png" /></figure><p>With the final state, you should have 3 cassandra containers up and running as a cluster. Now it’s time to understand the way to connect to that cluster. Move back to the cassandra-1 container on kitematic and go for the <em>Settings &gt; Ports</em> of that container. You should be seeing a screenshot given as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XjizmNn6dLfvw-8A-6ySsg.png" /></figure><p>So with our client, <a href="http://github.com/mulderbaba/cassandra-netty-rest-simple">cassandra-netty-rest-simple</a>, we’ll be accessing our cluster through IP: 192.168.99.100 and port: 32769, which is equivalent to 9042 port. Configuration of the client code should be done according to these values.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88fe71c2976f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Hibernate 5 on Payara]]></title>
            <link>https://medium.com/@mertcal/using-hibernate-5-on-payara-cc242212a5d6?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc242212a5d6</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Mon, 14 Sep 2015 22:17:36 GMT</pubDate>
            <atom:updated>2015-09-14T22:17:36.879Z</atom:updated>
            <content:encoded><![CDATA[<p>While deploying Hibernate 5.0.1.Final powered application on Payara 4.1.153, I faced with the stacktrace given below.</p><pre>java.lang.NoSuchMethodError: org.jboss.logging.Logger.debugf(Ljava/lang/String;I)V<br>at org.hibernate.internal.NamedQueryRepository.checkNamedQueries(NamedQueryRepository.java:149)<br>at org.hibernate.internal.SessionFactoryImpl.checkNamedQueries(SessionFactoryImpl.java:764)at org.hibernate.internal.SessionFactoryImpl.&lt;init&gt;(SessionFactoryImpl.java:495)<br>at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:444)</pre><p>With the hint of <strong>NoSuchMethodError</strong>, It was obvious that I was stuck with different versions of jboss-logging.jar so I executed <strong>mvn dependency:tree</strong> within my project to get the whole dependency tree. The result was given as follows for the hibernate jars that I used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7QGjxH-MVon_GbsF6iGGjw.png" /></figure><p>So the hibernate-core jar was transitively dependant on jboss-logging with version 3.3.0.Final. I excluded it with &lt;exclusion&gt; tag. The next execution of dependency:tree prompted me with another jboss-logging transitively bundled within hibernate-entitymanager.jar</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*YVu1banekqB22j-G9HkIsw.png" /></figure><p>I continued with excluding that also to see if any other transitive dependency exists. And yet there is more…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*F7YtPFckm7blpXeRP-vhQw.png" /></figure><p>Hibernate Validator came up with a slightly older version, 3.2.1.Final. Ok, moved on with excluding it. By the way Hibernate Validator team should update their transitive dependencies and align with the -core team IMO.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/918/1*AigMZL_fxndjg0bB-cRBfQ.png" /></figure><p>Ok, this was the last jboss-logging dependency that I came across. I was pretty sure that <strong>Hibernate 5</strong> depends on <strong>jboss-logging 3.3.X</strong>.</p><p>The error was mentioning about a missing <strong>debugf</strong> method, so after searching about it, I saw that the <em>…f</em> methods are newly implemented. There has to be an older version of jboss-logging on my classpath so I went through the jar files under Payara (a.k.a. Glassfish) installation folder.</p><p>I found a jboss-logging.jar under payara/glassfish/modules folder. When I opened up its manifest file I saw that the version was: <strong>3.1.3.GA</strong>.</p><p>Removing the jboss-logging.jar and adding the jboss-logging-3.3.0.Final.jar resolved the issue for me. I also added jboss-logging dependency with provided scope in pom.xml file. By adding a new dependency there was no need to exclude any artifact from hibernate-* dependency definitions.</p><pre>&lt;dependency&gt;<br>   &lt;groupId&gt;org.hibernate&lt;/groupId&gt;<br>   &lt;artifactId&gt;hibernate-core&lt;/artifactId&gt;<br>   &lt;version&gt;5.0.1.Final&lt;/version&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>   &lt;groupId&gt;org.hibernate&lt;/groupId&gt;<br>   &lt;artifactId&gt;hibernate-entitymanager&lt;/artifactId&gt;<br>   &lt;version&gt;5.0.1.Final&lt;/version&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>   &lt;groupId&gt;org.hibernate&lt;/groupId&gt;<br>   &lt;artifactId&gt;hibernate-validator&lt;/artifactId&gt;<br>   &lt;version&gt;5.2.1.Final&lt;/version&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>   &lt;groupId&gt;org.hibernate&lt;/groupId&gt;<br>   &lt;artifactId&gt;hibernate-jpamodelgen&lt;/artifactId&gt;<br>   &lt;version&gt;5.0.1.Final&lt;/version&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>   &lt;groupId&gt;org.jboss.logging&lt;/groupId&gt;<br>   &lt;artifactId&gt;jboss-logging&lt;/artifactId&gt;<br>   &lt;version&gt;3.3.0.Final&lt;/version&gt;<br>   &lt;scope&gt;<strong>provided</strong>&lt;/scope&gt;<br>&lt;/dependency&gt;</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc242212a5d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Helper method for waiting EC2 instances until they get ready]]></title>
            <link>https://medium.com/@mertcal/helper-method-for-waiting-ec2-instances-until-they-get-ready-1ca776f8693d?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ca776f8693d</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[android-app-development]]></category>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Tue, 18 Aug 2015 07:17:54 GMT</pubDate>
            <atom:updated>2015-08-18T07:17:54.907Z</atom:updated>
            <content:encoded><![CDATA[<p>Automating EC2 instance creation is a necessity when it comes to executing a series of <a href="https://medium.com/@mertcal/poc9-performant-setup-w-cassandra-netty-nginx-resteasy-to-handle-rest-web-service-requests-69332e74c63c">try-outs</a>. The method given below is a nice helper one that waits until your EC2 instances are ready before doing any heavy-load work.</p><pre><strong>private static void </strong>sleepUntilInstancesIsRunning(AmazonEC2 ec2, <br>          Set&lt;String&gt; instanceIdList) <strong>throws </strong>InterruptedException {<br>    <strong>while</strong>(<strong>true</strong>) {<br>        DescribeInstancesResult instanceStatusResult = <br>             ec2.describeInstances(<strong>new </strong>DescribeInstancesRequest().<br>             withInstanceIds(instanceIdList));<br><br>        <strong>if</strong>(Iterables.<em>all</em>(instanceStatusResult.getReservations(), <br>            <strong>new </strong>Predicate&lt;Reservation&gt;() {<br>               @Override<br>               <strong>public boolean </strong>apply(Reservation input) {<br>               <strong>return </strong>Iterables.<em>all</em>(input.getInstances(), <br>                  <strong>new </strong>Predicate&lt;Instance&gt;() {<br>                    @Override<br>                    <strong>public boolean </strong>apply(Instance input) {<br>                        <strong>return </strong>input.getState().getName().  <br>           equalsIgnoreCase(InstanceStateName.<strong><em>Running</em></strong>.toString());<br>                    }<br>                  });<br>            }<br>        })){<br>            <strong>break</strong>;<br>        } <strong>else </strong>{<br>            System.<strong><em>out</em></strong>.println(<strong>&quot;waiting...&quot;</strong>);<br>            Thread.<em>sleep</em>(1000);<br>        }<br>    }<br>}</pre><p>You can find the gist of the code in <a href="https://gist.github.com/anonymous/919c6345afb5c589d398">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ca776f8693d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improving the Quality on PDF size reducing of the Preview App on Mac]]></title>
            <link>https://medium.com/@mertcal/improving-the-quality-on-pdf-size-reducing-of-the-preview-app-on-mac-a505d30b493e?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/a505d30b493e</guid>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Thu, 06 Aug 2015 11:56:53 GMT</pubDate>
            <atom:updated>2015-08-06T13:16:32.033Z</atom:updated>
            <content:encoded><![CDATA[<p>Having a 1200dpi scanned PDF with a size of 20mb’s, I was looking for a way to shrink it down on Mac. Export feature on Preview app was my first option so I tried it by clicking File &gt; Export menu and then selecting the <em>Quartz Filter</em> as <strong>Reduce File Size</strong>.</p><p>The result was total crap. The file size reduced to 40kb from 20mb. The resoultion is way below than expected, probably around 30 or 40 dpi.</p><p>So while looking around to resolve this I found out that the reduce file size option could be configured by the file: <em>/System/Library/Filters/Reduce File Size.qfilter</em>.</p><p>The qfilter file is just an XML file and the key values <strong>Compression</strong> <strong>Quality</strong> (default value 0) and <strong>ImageSizeMax </strong>(default value 512)<strong> </strong>should be set bigger values to have better reducing quality. You can select the values according to the dpi value that you want to get. Table is ordered as dpi, compression quality, imagesizemax.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*vykHHy0uMNxq7oxrUoOU7g.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a505d30b493e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[PoC9] Performant setup w/ Cassandra, Netty, NGINX, RESTEasy to handle REST web service requests]]></title>
            <link>https://medium.com/@mertcal/poc9-performant-setup-w-cassandra-netty-nginx-resteasy-to-handle-rest-web-service-requests-69332e74c63c?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/69332e74c63c</guid>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Wed, 05 Aug 2015 13:26:51 GMT</pubDate>
            <atom:updated>2015-08-05T13:26:51.737Z</atom:updated>
            <content:encoded><![CDATA[<p>After a series of PoCs (<a href="https://medium.com/@mertcal/poc8-performance-tuning-on-cassandra-ec2-to-store-3-millions-minute-requests-by-retrieving-from-1c76ad0b8e62">this</a>, <a href="https://medium.com/@mertcal/poc6-serving-2-4-millions-minute-rest-web-service-requests-hosted-on-netty-proxied-through-nginx-7e6fb624d1ed">that</a> and <a href="https://medium.com/@mertcal/poc6-serving-2-4-millions-minute-rest-web-service-requests-hosted-on-netty-proxied-through-nginx-7e6fb624d1ed">many others</a>) executed on Amazon EC2, now it’s time to build up the stack that I’m planning to use for production. The setup goes like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KSgxDtgZHcWXiQ6rTmPJCw.png" /></figure><p>So there are 5 instances for creating the Cassandra cluster, 2 instances to host my custom REST web service implementation, 1 instance for the NGINX to load balance the requests and 1 instance for the requester client. You can find the custom implementations from given links: <a href="https://github.com/mulderbaba/http-requester">http-requester</a> and <a href="http://github.com/mulderbaba/cassandra-netty-rest-simple">cassandra-netty-rest-simple</a>.</p><p>Cassandra nodes <a href="https://medium.com/@mertcal/install-run-bare-bone-cassandra-cluster-on-ec2-d5f33884b225">are installed</a> and configured having the <em>commit log stored in a separate disk drive</em>. <a href="https://medium.com/@mertcal/install-nginx-on-ec2-instance-in-30-seconds-e714d427b01b">NGINX installation</a> is configured by editing the <em>/etc/sysctl.conf </em>file and adding <strong>net.ipv4.tcp_tw_reuse</strong> key with value <strong>1</strong>.</p><p>So http-requester creates requests and sends them to NGINX. NGINX load-balances them onto 2 servers that contains the REST web service impl. REST web services interacts with Cassandra cluster and store an object for each request. For this scenario the request/sec graph was as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_CM-_jrJPpXWHDRW0YeXBw.png" /></figure><p>For a total 6 minutes PoC handled <strong>16.680.508 requests</strong> with an average of <strong>46334 requests/sec</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69332e74c63c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[PoC8] Performance tuning on Cassandra & EC2 to store 3 millions/minute requests by retrieving from…]]></title>
            <link>https://medium.com/@mertcal/poc8-performance-tuning-on-cassandra-ec2-to-store-3-millions-minute-requests-by-retrieving-from-1c76ad0b8e62?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c76ad0b8e62</guid>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Tue, 04 Aug 2015 08:41:01 GMT</pubDate>
            <atom:updated>2015-08-04T08:41:01.800Z</atom:updated>
            <content:encoded><![CDATA[<h3>[PoC8] Performance tuning on Cassandra &amp; EC2 to store 3 millions/minute requests by retrieving from a Netty-based REST web service requests</h3><p>I’ve already implemented <a href="https://medium.com/@mertcal/poc7-storing-2millions-minute-records-in-cassandra-by-retrieving-from-a-netty-based-rest-web-81733302dabb">a working PoC for storing 2 millions requests a minute on Cassandra</a> but the final graph that was crafted from result numbers showed me that configuration behaves badly.</p><p>So while I was looking around to find a workaround, I came across with <a href="http://jonathanhui.com/cassandra-performance-tuning-and-monitoring">this awesome blog</a> stating that : <em>The biggest performance gain for write is to put commit log in a separate disk drive</em>.</p><p>So I was convinced to add a new SSD to each node in my Cassandra Cluster. This means that I need to add another volume on EC2. To handle the creation of another volume, I added an SSD with Provisioned IOPS (see image below for details) while launching my instance. The IOPS was set to 600, I think I’ll try the max value 3000 with an upcoming PoC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IZ5cBINrcREkUcR32YGBGg.png" /></figure><p>To make this newly added Amazon EBS volume available to the Linux installation, I needed to format it first and then mount it. Just follow the steps detailed at <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html">here</a> to handle it. They’re well defined and precise. Kudos to Amazon Developer Team…</p><p>After adding the new volume, it was time to change the configuration in <em>cassandra.yaml</em> for having the commitlogs stored in my new volume. I stated my <strong><em>mount_point </em></strong>as <strong>/commitlog </strong>for the new drive, you can do the same for sure.</p><pre>commitlog_directory: /commitlog</pre><p>The rest for getting the Cassandra cluster up and running was same as it had been done in the <a href="https://medium.com/@mertcal/poc7-storing-2millions-minute-records-in-cassandra-by-retrieving-from-a-netty-based-rest-web-81733302dabb">previous try-out</a>, so I tested the same scenario with the same EC2 instance setup. Results were %50 better than the previous try-out with an average of 50k/sec.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rT4CosFxEG86zby1htDDlg.png" /></figure><p>The final point is: While working with Cassandra Cluster, SSD’s should be tailored with regarding configurations.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c76ad0b8e62" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[PoC7] Storing ~2millions/minute records in Cassandra by retrieving from a Netty-based REST web…]]></title>
            <link>https://medium.com/@mertcal/poc7-storing-2millions-minute-records-in-cassandra-by-retrieving-from-a-netty-based-rest-web-81733302dabb?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/81733302dabb</guid>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Tue, 28 Jul 2015 16:41:58 GMT</pubDate>
            <atom:updated>2015-07-30T07:39:32.393Z</atom:updated>
            <content:encoded><![CDATA[<h3>[PoC7] Storing ~2millions/minute records in Cassandra by retrieving from a Netty-based REST web service requests</h3><p>Having delt with <a href="https://medium.com/@mertcal/poc6-serving-2-4-millions-minute-rest-web-service-requests-hosted-on-netty-proxied-through-nginx-7e6fb624d1ed">REST web service implementations based on RESTEasy and Netty</a> for quite some time, I’ve managed to handle from 20k/sec to 40k/sec requests in on EC2 instances. But without storing any of those requests, that won’t make any sense for my production setup. So it’s time to move onto the NoSQL land. I sticked with <a href="http://cassandra.apache.org/">Cassandra</a>, which offers columnar DB with scalability and availability features.</p><p>I installed Cassandra Cluster on 5 nodes and got it up &amp; running. You can find the steps to do it in <a href="https://medium.com/@mertcal/install-run-bare-bone-cassandra-cluster-on-ec2-d5f33884b225">here</a>. For connecting to Cassandra, I’ve modified my <a href="http://github.com/mulderbaba/netty-rest-simple">netty-rest-simple</a> REST web service client and named it as <a href="https://github.com/mulderbaba/cassandra-netty-rest-simple">cassandra-netty-rest-simple</a>. It uses <a href="https://github.com/doanduyhai/Achilles">Achilles</a> under the hood, which is an ORM-like persistence manager for Cassandra. Other than that it uses RESTEasy for JAX-RS compliant REST web service implementation and Netty for handling the high-throughput with its NIO architecture.</p><p>I used the AMI <strong>ami-a8221fb5 </strong>(from Frankfurt region)<strong> </strong>and instance types <strong>i2.2xlarge</strong> and <strong>m4.2xlarge </strong>respectively for creating all 7 instances given in the diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FyaYEcKuQ0FhXRX6Mepr2A.png" /></figure><p>After setting the Cassandra cluster, just execute the <em>nodetool status</em> on one of the nodes and you should be getting an output as seen below. I omitted the Host ID and Rack columns from the output for simplicity.</p><pre>Datacenter: datacenter1<br>=======================<br>Status=Up/Down<br>|/ State=Normal/Leaving/Joining/Moving<br> — Address Load      Tokens Owns (effective)<br>UN IP1     51.63 KB  256    42.9% <br>UN IP2     66.51 KB  256    37.9% <br>UN IP3     82.5 KB   256    39.4% <br>UN IP4     66.58 KB  256    38.4% <br>UN IP5     104.86 KB 256    41.4%</pre><p>To get <a href="http://github.com/mulderbaba/cassandra-netty-rest-simple">cassandra-netty-rest-simple</a> (instance F) up and running just execute the command given as follows, after creating a fat jar out of it of course with the maven-shade-plugin defined. The each node’s IP of the Cassandra cluster can be provided as parameter with a space delimited list.</p><pre>java -jar cassandra-netty-rest-simple-0.0.1.jar IP1 IP2 IP3 IP4 IP5</pre><p>Within the connector part of the code, the replication factor was set to <strong>3</strong> and the consistency level was set to <strong>ONE</strong> for both read and write. You can just have a peek at the code to get a basic understanding.</p><p>Execution of the <a href="https://github.com/mulderbaba/http-requester">http-requester</a> was same as the previous PoCs, <a href="https://medium.com/@mertcal/poc6-serving-2-4-millions-minute-rest-web-service-requests-hosted-on-netty-proxied-through-nginx-7e6fb624d1ed">here</a> and <a href="https://medium.com/@mertcal/poc5-handling-20k-sec-requests-on-a-netty-based-resteasy-web-service-proxied-through-nginx-134988b3c415">there</a>.</p><pre>java -jar http-requester-0.0.1.jar &lt;IP&gt; &lt;PORT&gt; 1000000 1000000 200</pre><p>A sampling for a minute gave out the numbers which sums up to <strong>1.675.121 requests.</strong></p><pre>22690 req/sec<br>31063 req/sec<br>31557 req/sec<br>26515 req/sec<br>27138 req/sec<br>31510 req/sec<br>22223 req/sec<br>29160 req/sec<br>31140 req/sec<br>29701 req/sec<br>26529 req/sec<br>31512 req/sec<br>27290 req/sec<br>26686 req/sec<br>21130 req/sec<br>31218 req/sec<br>31766 req/sec<br>26539 req/sec<br>31290 req/sec<br>26965 req/sec<br>29155 req/sec<br>23700 req/sec<br>24841 req/sec<br>31416 req/sec<br>26128 req/sec<br>31541 req/sec<br>31692 req/sec<br>21518 req/sec<br>31582 req/sec<br>26209 req/sec<br>26203 req/sec<br>25848 req/sec<br>31205 req/sec<br>31477 req/sec<br>22138 req/sec<br>26038 req/sec<br>30692 req/sec<br>25506 req/sec<br>25211 req/sec<br>25685 req/sec<br>27375 req/sec<br>30977 req/sec<br>26741 req/sec<br>29672 req/sec<br>26129 req/sec<br>31119 req/sec<br>26148 req/sec<br>31313 req/sec<br>27776 req/sec<br>26157 req/sec<br>28272 req/sec<br>25946 req/sec<br>30600 req/sec<br>23647 req/sec<br>29033 req/sec<br>26308 req/sec<br>30148 req/sec<br>26429 req/sec<br>29779 req/sec<br>30145 req/sec</pre><p>Fair enough but since Amazon creates a bottleneck on the network with its 1Gpbs bandwith, I thought I should fire up <strong>m4.10XLarge</strong> machines instead of <strong>m4.2xLarge</strong> ones :). 40 cores but 10Gbps network, yay!. Having this configuration running for a minute ended up <strong>2.375.654</strong> requests/min in numbers. Yeah better but not good enough... The graph of the requests/sec were as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vheCht14gEwFjg_GtRpKgA.png" /></figure><p>I see numbers as high as <strong>70k/sec</strong> but the whole replay spikes badly. Looks like I need to take care of some sort of drain. Stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81733302dabb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Install & run bare-bone Cassandra Cluster on EC2]]></title>
            <link>https://medium.com/@mertcal/install-run-bare-bone-cassandra-cluster-on-ec2-d5f33884b225?source=rss-56c3e2f1314c------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5f33884b225</guid>
            <dc:creator><![CDATA[Mert Çalışkan]]></dc:creator>
            <pubDate>Mon, 27 Jul 2015 14:04:02 GMT</pubDate>
            <atom:updated>2015-07-27T21:55:29.973Z</atom:updated>
            <content:encoded><![CDATA[<p>I’ve already created a <a href="https://medium.com/@mertcal/running-cassandra-cluster-on-docker-d9a44aafebb9">Cassandra Cluster on EC2 by employing Docker</a> but if you need to do a bare-bone installation to compare some performance metrics without hassling with any containers, here is how you can achieve it.</p><p>Cassandra v2.1.8 encourages us to use JDK8 but the AMI with id: <strong>ami-a8221fb5</strong> that I used comes with OpenJDK7. <a href="https://medium.com/@mertcal/easily-install-oracle-jdk8-on-your-ec2-instance-9317644a42fa">So first follow the steps in here to upgrade to JDK8 in your EC2 instance</a>.</p><p>Then you can continue to create/edit-if-exists the <em>datastax.repo</em> file with the command given as follows:</p><pre>sudo vi /etc/yum.repos.d/datastax.repo</pre><p>The content of the <em>datastax.repo </em>file should be as follows:</p><pre>[datastax] <br>name = DataStax Repository<br>baseurl = <a href="http://rpm.datastax.com/community">http://rpm.datastax.com/community</a><br>enabled = 1<br>gpgcheck = 0</pre><p>Install Cassandra with the command given as follows. Press ‘y’ whenever prompted.</p><pre>sudo yum install dsc21</pre><p>To check if the installation is a success, execute:</p><pre>/etc/init.d/cassandra status</pre><p>You should be seeing the output: <em>cassandra is stopped</em>.</p><p>Before starting cassandra service, we need to configure it for our cluster. The configuration file resides as <strong>/etc/cassandra/conf/cassandra.yaml</strong>. A sample configuration is given below. The bold parts are the added/modified parts of the default configuration file. The <strong>listen_address</strong> and the <strong>seed_provider</strong> part already exists in the default configuration file. So if you are going the change the file via <em>vi</em> for instance, better to remove them and add them to the top along with the other entries.</p><p>The <strong>listen_address</strong> sholuld be the private IP address of the instance. The <strong>broadcast_address</strong>, <strong>broadcast_rpc_address</strong> should be the public IP address of the instance. The <strong>seeds </strong>should refer to the public IP addresses of all the seeds and those IP values can be separated with commas. The whole value should be wrapped by quotation marks. If we have 3 nodes for a cluster and one of them is the seed node, configuration for all the 3 nodes should refer to the same public IP address of that seed node.</p><pre><strong>listen_address: &lt;private-ip-address-of-instance&gt;<br>broadcast_address: &lt;public-ip-address-of-instance&gt;<br>rpc_address: 0.0.0.0<br>broadcast_rpc_address: &lt;public-ip-address-of-instance&gt;<br>seed_provider:<br> - class_name: org.apache.cassandra.locator.SimpleSeedProvider<br>   parameters:<br>   - seeds: “&lt;public-ip-address-of-seed-instance&gt;”</strong></pre><pre>cluster_name: ‘Test Cluster’<br>num_tokens: 256<br>hinted_handoff_enabled: true<br>max_hint_window_in_ms: 10800000<br>hinted_handoff_throttle_in_kb: 1024<br>max_hints_delivery_threads: 2<br>batchlog_replay_throttle_in_kb: 1024</pre><pre>authenticator: AllowAllAuthenticator<br>authorizer: AllowAllAuthorizer<br>permissions_validity_in_ms: 2000</pre><pre>partitioner: org.apache.cassandra.dht.Murmur3Partitioner<br>data_file_directories:<br> - /var/lib/cassandra/data</pre><pre>commitlog_directory: /var/lib/cassandra/commitlog<br>disk_failure_policy: stop<br>commit_failure_policy: stop<br>key_cache_size_in_mb:<br>key_cache_save_period: 14400<br>row_cache_size_in_mb: 0<br>row_cache_save_period: 0<br>counter_cache_size_in_mb:<br>counter_cache_save_period: 7200<br>saved_caches_directory: /var/lib/cassandra/saved_caches</pre><pre>commitlog_sync: periodic<br>commitlog_sync_period_in_ms: 10000<br>commitlog_segment_size_in_mb: 32<br>concurrent_reads: 32<br>concurrent_writes: 32<br>concurrent_counter_writes: 32</pre><pre>memtable_allocation_type: heap_buffers<br>index_summary_capacity_in_mb:<br>index_summary_resize_interval_in_minutes: 60</pre><pre>trickle_fsync: false<br>trickle_fsync_interval_in_kb: 10240</pre><pre>storage_port: 7000<br>ssl_storage_port: 7001<br>start_native_transport: true<br>native_transport_port: 9042<br>start_rpc: true<br>rpc_port: 9160<br>rpc_keepalive: true<br>rpc_server_type: sync</pre><pre>thrift_framed_transport_size_in_mb: 15<br>incremental_backups: false<br>snapshot_before_compaction: false<br>auto_snapshot: true</pre><pre>tombstone_warn_threshold: 1000<br>tombstone_failure_threshold: 100000<br>column_index_size_in_kb: 64<br>batch_size_warn_threshold_in_kb: 5</pre><pre>compaction_throughput_mb_per_sec: 16<br>compaction_large_partition_warning_threshold_mb: 100<br>sstable_preemptive_open_interval_in_mb: 50</pre><pre>read_request_timeout_in_ms: 5000<br>range_request_timeout_in_ms: 10000<br>write_request_timeout_in_ms: 2000<br>counter_write_request_timeout_in_ms: 5000<br>cas_contention_timeout_in_ms: 1000<br>truncate_request_timeout_in_ms: 60000<br>request_timeout_in_ms: 10000<br>cross_node_timeout: false<br>endpoint_snitch: SimpleSnitch</pre><pre>dynamic_snitch_update_interval_in_ms: 100<br>dynamic_snitch_reset_interval_in_ms: 600000<br>dynamic_snitch_badness_threshold: 0.1</pre><pre>request_scheduler: org.apache.cassandra.scheduler.NoScheduler<br>server_encryption_options:<br>internode_encryption: none<br>keystore: conf/.keystore<br>keystore_password: cassandra<br>truststore: conf/.truststore<br>truststore_password: cassandra</pre><pre>client_encryption_options:<br>enabled: false<br>keystore: conf/.keystore<br>keystore_password: cassandra</pre><pre>internode_compression: all<br>inter_dc_tcp_nodelay: false</pre><p>If you are lazy like me, instead of editing the cassandra.yaml file via <em>vi</em>, you can upload it directly via <em>scp </em>as below. If you encounter any permission errors, just <em>chmod</em> the yaml file before overwriting it.</p><pre>scp -i /path/to/myAwesome.pem -oStrictHostKeyChecking=no /path/to/cassandra.yaml ec2-user@&lt;public-ip-address-of-instance&gt;:/etc/cassandra/conf/cassandra.yaml</pre><p>Now we are ready to start the seed node:</p><pre>sudo service cassandra start</pre><p>When you execute <strong>nodetool status</strong> you should be seeing something like:</p><pre>Datacenter: datacenter1<br>=======================<br>Status=Up/Down<br>|/ State=Normal/Leaving/Joining/Moving<br> — Address Load Tokens Owns Host ID Rack<br>UN &lt;IP&gt; 168.97 MB 256 ? 45efc448–8c7b-4511–9057–5d2b62ae7190 rack1</pre><p>Adding a new node is a piece of cake, just modify the previously given <strong>cassandra.yaml</strong> with its IP and start the cassandra service. After executing the service, you should be seeing 2 nodes when you execute <em>nodetool status</em>.</p><pre>Datacenter: datacenter1<br>=======================<br>Status=Up/Down<br>|/ State=Normal/Leaving/Joining/Moving<br> — Address Load Tokens Owns Host ID Rack<br>UN &lt;IP1&gt; 168.97 MB 256 ? 45efc448–8c7b-4511–9057–5d2b62ae7190 rack1<br>UN &lt;IP2&gt; 169.28 MB 256 ? bf98b5bc-537f-4c05-ac3d-0a428001211d rack1</pre><p>If you’d like to use the CLI for running CQL queries, execute:</p><pre>cqlsh -u cassandra -p cassandra</pre><p>You can continue by configuring more nodes to join the cluster.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5f33884b225" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>