<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by İbrahim Gürses on Medium]]></title>
        <description><![CDATA[Stories by İbrahim Gürses on Medium]]></description>
        <link>https://medium.com/@ibrahimgrses?source=rss-792d45982230------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*tnsQUOhiwfDYr24F.jpeg</url>
            <title>Stories by İbrahim Gürses on Medium</title>
            <link>https://medium.com/@ibrahimgrses?source=rss-792d45982230------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:04:20 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ibrahimgrses/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[What Node.js Custom Runtime Brings and How We Developed It?]]></title>
            <link>https://medium.com/@ibrahimgrses/what-node-js-custom-runtime-brings-and-how-we-developed-it-94d93a0f2262?source=rss-792d45982230------2</link>
            <guid isPermaLink="false">https://medium.com/p/94d93a0f2262</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[serverless-monitoring]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[İbrahim Gürses]]></dc:creator>
            <pubDate>Sat, 26 Jan 2019 08:33:55 GMT</pubDate>
            <atom:updated>2019-01-26T08:33:55.638Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oYiwAYxrEoTWSxhNASsCog.png" /></figure><p>“Serverless monitoring with no code change and zero overhead” is our mantra at Thundra. I know it is a bold statement, but we have been working hard to fulfill this claim. First, we achieved “zero overhead” by leveraging <a href="https://medium.com/thundra/how-to-monitor-aws-lambda-with-zero-overhead-790e4d78ff8f">async monitoring</a> and even today Thundra is still the only agent-based solution which supports async monitoring. Then, we bring <a href="https://medium.com/thundra/detailed-serverless-monitoring-using-a-completely-automated-approach-c148fa8cfa47">automatic monitoring</a> support to our NodeJS agent in which our <a href="https://github.com/thundra-io/serverless-plugin-thundra">serverless plugin</a> automatically wraps your AWS Lambda functions and you can deploy your functions without changing anything in your code. Even with these nice and convenient ways, what if there was a simpler configuration to enable monitoring features of Thundra on an already deployed AWS Lambda function with just a few changes over AWS Lambda Console. AWS Lamba Layers and Custom Runtime comes to the rescue.</p><h3>TL;DR;</h3><p>Thundra NodeJS Custom Runtime lets you monitor your AWS Lambda Functions with no code change. Check <a href="https://docs.thundra.io/docs/node-custom-runtime-and-layer-support#section--how-to-use-thundra-node-layer-and-custom-runtime-">Thundra Docs</a> for detailed instructions. You can also check out <a href="https://www.youtube.com/watch?v=Zz1t8lzRZIM">this video</a> which showcases how you can integrate Thundra with your AWS Lambda function in less than a minute.</p><h3>AWS Lambda Layers and Custom Runtime</h3><p>AWS Re:Invent 2018 has come and gone with exciting announcements for the serverless community. We were particularly more excited for <a href="https://aws.amazon.com/tr/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/">Layer and Bring Your Own Runtime</a> support because it could enable us to deliver Thundra NodeJS agent with literal “no code change” approach. Thundra was one of the <a href="https://thenewstack.io/serverless-thundra-adds-observability-to-aws-lambda/">launch partners</a> for this announcement, which was a thrilling moment for us because we were able to play with this technology beforehand. For those who do not know what layers and custom runtime is :</p><ul><li><strong>Lambda Layers</strong>, a way to centrally manage code and data that is shared across multiple functions.</li><li><strong>Lambda Runtime API</strong>, a simple interface to use any programming language, or a specific language version, for developing your functions.</li></ul><h3><strong>So How We Developed It?</strong></h3><p>So, in order to develop Custom Runtime for NodeJS, we needed to learn how the current NodeJS runtime of AWS Lambda worked. For those who are curious like <a href="https://dzone.com/articles/inside-a-lambda-runtime-a-peek-into-the-serverless">Janaka Bandara</a>, you already know you can run shell commands in the AWS Lambda and learn about the underlying OS Level stuff. With a few lines of NodeJS code like below, you can run a shell command inside AWS Lamda and get its output.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2b30b0210d994ea77d0ab1a93046e837/href">https://medium.com/media/2b30b0210d994ea77d0ab1a93046e837/href</a></iframe><p>You can run the code above on AWS Lambda, it is a simple lambda function which gets a command and forks a child process then executes the command and finally logs the output to the console. For example, if you invoke this lambda function with JSON payload<strong><em> `{“cmd”: “pwd”}</em></strong>`, it will log <strong><em>`/var/task`</em></strong> to the console. In Linux systems,<strong><em>`pwd`</em></strong> command prints the current working directory. So we can see that AWS Lambda NodeJS runtime runs the user code under <strong><em>`/var/task`</em></strong> folder.</p><p>Now, with the help of running simple shell commands, we could see how the current AWS NodeJS runtime operates, how it parses the handler, and what it does for initialization, what are the environment variable in the container. We extracted the `index.js` file which AWS Lambda uses to run user functions. It taught us a lot about how AWS builds Lambda Context object which environment variables are used, how the execution loop is constructed. Sure there were some native calls to some C++ add-ons impairing our ability to analyze the code, but it gave us a general idea for developing our custom runtime. You can take a look at this code from <a href="https://gist.github.com/kobalski/e7115a01c924f03e6ee29cd3a62c5059">here</a>.</p><p>Then, we started developing our custom runtime, but we had a major problem. Current Custom Runtime only included NodeJS Version 4.3 which is an archaic version of NodeJS and our agent was not designed to work on that old version of NodeJS. So, we needed a NodeJS binary. AWS Lambda runs Custom Runtime on Amazon Linux with this <a href="https://console.aws.amazon.com/ec2/v2/home#Images:visibility=public-images;search=amzn-ami-hvm-2017.03.1.20170812-x86_64-gp2">AMI</a> specifically. So, we started an EC2 instance with that AMI and built NodeJS from source code on that instance. We created a node binary for NodeJS version 8.10 for Thundra Custom Runtime, we could include the latest version of NodeJS, but we did not have time to test our agent with the latest version. So, we included it with 8.10 but we can provide a layer with the latest versions of NodeJS in the future.</p><p>After finishing the implementation of our Custom Runtime, we packaged everything as an AWS Lambda Layer so that user can simply add our runtime as a layer and start using Thundra. The final version of the layer included these files:</p><p><strong><em>| — bootstrap # entry point for custom runtime, starts index.js<br>| — node<br>| | — index.js # Custom Runtime Implementation <br>| | — node # Node 8.10 binary, used by Custom Runtime <br>| | — node_modules # Contains Thundra NodeJS agent dependencies <br>| | — package-lock.json # Contains Thundra NodeJS agent build info <br>| | — package.json # Contains Thundra NodeJS agent build info</em></strong></p><p>So, how did we implement the custom runtime? We followed the steps in the official AWS <a href="https://docs.aws.amazon.com/en_us/lambda/latest/dg/runtimes-custom.html">doc</a>, as an extra step, we are automatically wrapping your functions with Thundra before running your code so that you don’t need to wrap your code with our handler at all. Our custom runtime implementation consists of the steps like below.</p><ol><li><strong>Get an event</strong> — Call the <a href="https://docs.aws.amazon.com/en_us/lambda/latest/dg/runtimes-api.html#runtimes-api-next">next invocation</a> API to get the next event. The response body contains the event data. Response headers contain the request ID and other information.</li><li><strong>Create a context object</strong> — Create an object with context information from environment variables and headers in the API response.</li><li><strong>Wrap user function with Thundra Agent </strong>— Thundra agent uses wrapping for monitoring with custom runtime we do that for you now.</li><li><strong>Invoke the wrapped function handler</strong> — Pass the event and context object to the handler wrapped with Thundra</li><li><strong>Handle the response</strong> — Call the <a href="https://docs.aws.amazon.com/en_us/lambda/latest/dg/runtimes-api.html#runtimes-api-response">invocation response</a> API to post the response from the handler.</li><li><strong>Handle errors</strong> — If an error occurs, call the <a href="https://docs.aws.amazon.com/en_us/lambda/latest/dg/runtimes-api.html#runtimes-api-invokeerror">invocation error</a> API</li><li><strong>Cleanup</strong> — Release unused resources, send data to other services, or perform additional tasks before getting the next event.</li><li><strong>Wait for the Next Invocation — </strong>Custom Runtime freezes the container and thaws the container when a new request comes and we go to step 1 again and continue the loop</li></ol><p>In order to visualize things better, the below diagram shows what happens during the first invocation that comes to the Custom Runtime container and Runtime API initializes everything. The part annotated with Cold Start happens when each time AWS Lambda creates a brand new container.</p><p>The part where we annotate with invoke phase is run inside a loop, for each new invocation coming to already alive container runs the invoke phase again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/532/1*YnEPxaacIiiyyDCfWat23g.png" /><figcaption>Figure 1: Sequence diagram of what happens when the first request comes</figcaption></figure><h3><strong>No code change</strong></h3><p>Layers and Runtime API are two powerful features offered by AWS Lambda. It enabled us to run the Thundra monitoring agent with no code change on a serverless platform. In this blog post, I tried to a give a walkthrough on how we developed Thundra Custom Runtime by explaining the inner workings of Custom Runtime, and how we use Layer to offer our NodeJS agent. The value of AWS Lambda is its simplicity and how it abstracts underlying platform stuff. Custom Runtime may be against to that values. At the end of the day, the beauty of serverless is to focus on our business problems, not the platforms or runtimes they run. But for those who are developing services for AWS Lambda like us, these are fantastic technologies which will bring many cool things to the serverless ecosystem.</p><p>You can try Thundra NodeJS Custom Runtime by following the simple instructions on the <a href="https://docs.thundra.io/docs/node-custom-runtime-and-layer-support#section--how-to-use-thundra-node-layer-and-custom-runtime-">Thundra Docs.</a> You can also check out <a href="https://www.youtube.com/watch?v=Zz1t8lzRZIM">this video</a> which showcases how you can integrate Thundra with your AWS Lambda function in less than a minute. Thundra is free to use and set up takes only a few minutes. <a href="https://www.thundra.io/sign-up">Get your account</a> and start using Thundra today.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=94d93a0f2262" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Look inside an Alexa Skill with Thundra]]></title>
            <link>https://medium.com/@ibrahimgrses/look-inside-an-alexa-skill-with-thundra-9263c7fd5e0c?source=rss-792d45982230------2</link>
            <guid isPermaLink="false">https://medium.com/p/9263c7fd5e0c</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[alexa]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[İbrahim Gürses]]></dc:creator>
            <pubDate>Fri, 17 Aug 2018 13:52:22 GMT</pubDate>
            <atom:updated>2018-08-17T13:52:22.529Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pB4TmvG1vBOR0weQc5gFWw.jpeg" /></figure><p>Alexa is Amazon’s virtual assistant and the brain behind tens of millions of Echo devices like the Echo Show and Echo Spot. Alexa provides capabilities, (called skills), that enable customers to experience more personalized service. The<a href="https://www.amazon.com/alexa-skills/b?ie=UTF8&amp;node=13727921011"> Alexa Skills Store </a>currently has more than 45,000 published skills and this number is increasing rapidly.</p><p>Amazon recommends using AWS Lambda to develop the backend APIs for custom skills. So right now, the fastest way to publish your cool custom skill idea is to leverage serverless technologies! This is great news for us at Thundra because we can have some fun showing you how you can use Thundra to monitor and observe your Alexa Skills.</p><p>In fact, today I’m going to share how I developed a custom Alexa Skill with the Node.js Alexa Skill Kit (ASK). I then used Thundra to manually instrument my Lambdas, allowing me to understand what happens while during function brings me the information about Ice and Fire.</p><p>Manual instrumentation is useful when you want to examine any specific piece of code that you suspect is causing problems. It’s a very flexible and straightforward approach as you can simply add a custom variable wherever you need to start and finish a span.</p><p>Let’s dive in to see how it helps to identify the flow throughout the life of a Alexa query.</p><h3>Yet Another Game Of Thrones Wiki</h3><p>Designing a natural Voice User Interface (VUI) is a difficult task as it requires mastery of your target language, so I thought I should make my demo app simple. To do that, I decided to make a simple Alexa Skill that answers some fundamental questions about a TV show. I searched for public APIs for some of my favorite TV Shows and I found <a href="https://anapioficeandfire.com/"><em>“An API of Ice And Fire</em></a>,”. This API is <a href="https://anapioficeandfire.com/About">described as</a>, “<em>The world’s greatest source for quantified and structured data from the universe of Ice and Fire</em>.” Perhaps building a simple Game of Thrones wiki is not the most original idea, but I think it is a good starter project.</p><p>To design a Voice User Interface (VUI) for Alexa, you need to map the user’s spoken input to intents. An <strong><em>intent</em></strong> represents an action that fulfills a user’s spoken request. Intents can contain sample <strong><em>utterances</em></strong> and optionally have arguments called <strong><em>slots</em></strong>. I will explain these concepts with examples but you can also check out the Interaction Model <a href="https://developer.amazon.com/docs/custom-skills/create-the-interaction-model-for-your-skill.html">docs</a> for more details.</p><p>Game of Thrones Wiki will have 3 basic intents:</p><ul><li>`<strong><em>WhoPlayedCharacterIntent</em></strong>`: This intent answers who plays a certain character on the TV show</li><li>`<strong><em>BookInfoIntent</em></strong>`: This intent gives information about the Fire and Ice book series</li><li>`<strong><em>HouseOverlordIntent</em></strong>`. This intents answers who the current overlord of a house is.</li></ul><p>For each intent, we need to provide sample utterances and define slots for each utterance.</p><p>For example, if the user says ‘<strong><em>Who played Arya Stark?</em></strong>’, Alexa will match speech input to `<strong><em>WhoPlayedCharacterIntent</em></strong>` and match the `<em>Arya Stark</em>` with the slot `<strong><em>character</em></strong>`, because `<strong><em>WhoPlayedCharacterIntent</em></strong>` contains a sample utterance `<strong><em>who played {character}</em></strong>`. Once intent creation was complete, my console looked like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*CfG68Tx02j5Ir9Q9" /><figcaption>Developer Console for Custom Alexa Skills</figcaption></figure><h3>Trace your Lambda with Thundra</h3><p>According to the <a href="https://serverless.com/blog/2018-serverless-community-survey-huge-growth-usage/">2018 Serverless Community Survey</a>, the top 3 challenges with serverless development are debugging, monitoring, and testing. So, during development, I added Thundra’s manual instrumentation and log support to my Lambdas to debug my application.</p><p>Let’s start writing our Lambda function for the backend API and integrate it with Thundra.</p><p>First, initialize an empty Node.js project and grab the Alexa Skill Kit, Thundra core, and Thundra log plugins from the NPM registry with the following command.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6ac1daa45b10b7b3bc2b58aa11eb1c6e/href">https://medium.com/media/6ac1daa45b10b7b3bc2b58aa11eb1c6e/href</a></iframe><p>Before writing our intent handlers, import the Alexa Skill Kit SDK and the Thundra agent. With the `<strong><em>skillBuilder</em></strong>`, build your Lambda function and wrap it with the Thundra agent. You don’t need to worry about `<strong><em>ErrorHandler</em></strong>` and `<strong><em>WhoPlayedCharacterIntentHandler</em></strong>`right now.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ace246518f5aecb55b6ecbf5e79f77dc/href">https://medium.com/media/ace246518f5aecb55b6ecbf5e79f77dc/href</a></iframe><p>Now we can write our first intent handler and add trace information. The Thundra agent is OpenTracing compliant. We can use the <a href="https://doc.esdoc.org/github.com/opentracing/opentracing-javascript/">OpenTracing API</a> and the Thundra tracer to measure execution times for specific pieces of code.</p><p>To write an `<strong><em>IntentHandler</em></strong>`, we need to implement two functions:</p><ul><li>First one is `<strong><em>canHandle()</em></strong>` function. In the Alexa Skill Kit SDK, the request routing is based on a “<em>Can you handle this?</em>” concept, which means that every handler lays out the specific condition(s) that it is capable of handling and returns a `<strong><em>boolean</em></strong>` response to requests. You can set these conditions within the `<strong><em>canHandle()</em></strong>` function defined inside the handler. The `<strong>c<em>anHandle()</em></strong>` function is required in every handler.</li><li>The second function we need to implement is the `<strong><em>handle</em></strong>` function. If the condition(s) set in the `<strong><em>canHandle()</em></strong> ` function returns `<strong><em>true</em></strong>`, the code included in the `<strong><em>handle()</em></strong>` function is executed.</li></ul><p>In order to instrument `<strong><em>WhoPlayedCharacterIntentHandler</em></strong>` with Thundra monitoring, we do the following:</p><ul><li>At the beginning of `<strong><em>handle()</em></strong>` function of `<strong><em>WhoPlayedCharacterIntentHandler</em></strong>`, we grab `<strong><em>ThundraTracer</em></strong>`</li><li>Then, start a span with name `’<strong><em>WhoPlayedCharacterIntent</em></strong>’`</li><li>Make our API call</li><li>Add `<strong><em>speechText</em></strong>` as a tag to the span before the return of `<strong><em>handle()</em></strong>` function</li><li>Finally, we finish the span.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/958c5306f30a16f7902170276bd4fffb/href">https://medium.com/media/958c5306f30a16f7902170276bd4fffb/href</a></iframe><p>Errors can happen while coding, so it is important how we handle and report these errors. You can add a `<strong><em>ErrorHandler</em></strong>` function to the Alexa Skill Kit SDK which is a fallback mechanism for unhandled intents and errors. If any of the intent handlers throws an error, the error is caught by `<strong><em>ErrorHandler</em></strong>`. In the error handler, we log the exceptions to both CloudWatch and the Thundra log plugin as well as the current active span.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e952d49567cf626df6340cfa975794f3/href">https://medium.com/media/e952d49567cf626df6340cfa975794f3/href</a></iframe><p>Now, we need to deploy our Lambda function and integrate with Alexa. To deploy your Lambda function, you can use the serverless template on the <a href="https://github.com/thundra-io/thundra-examples-lambda-nodejs/blob/master/alexa-skill-demo/lambda/custom/serverless.yml">thundra-examples-lambda-nodejs</a> repo. You will need to make necessary changes in the template and before deploying your function with <a href="https://serverless.com/">The Serverless Framework.</a></p><p>As you can see below, the `<strong><em>WhoPlayedCharacterIntentHandler</em></strong>` works as expected. Let’s look at Thundra to figure out what happened during the invocation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Vtzmj9jjMa6FZQ0g" /><figcaption>Alexa Simulator for Testing</figcaption></figure><p>On the detailed invocation page of the Thundra Web Console, we can see metrics about our function also we can directly jump to CloudWatch logs. Most importantly, we can look at the spans that we created manually inside the code. As you can see, the first span is the Lambda invocation and the second span is `<strong><em>WhoPlayedCharacterIntent</em></strong>`, and then our intent makes an `<strong><em>API Call</em></strong>` which takes the longest time. Additionally, when you click each span, you can see the tags we added inside the code. You can see that the tag `<strong><em>speechText</em></strong>` is the return value for our skill. Adding manual instrumentation to code during development helps developer to detect any problems earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RWjWAGzOAbOyLJ3e" /><figcaption>Thundra Invocation Detail with Manual Instrumentation</figcaption></figure><p>My full code is available on <a href="https://github.com/thundra-io/thundra-examples-lambda-nodejs/tree/master/alexa-skill-demo">GitHub</a>. You can try it yourself and play with it! Just <a href="https://console.thundra.io/signup?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=nodejs_manual_instrumentation&amp;utm_content=code_example">sign up for our free beta</a> and follow the instructions to deploy it into your AWS environment.</p><p>I am excited about Thundra’s new <a href="https://docs.thundra.io/v1.0/docs/manual-instrumentation-with-open-tracing-api">Node.js manual instrumentation</a> functionality! I hope it will help you detect, debug, and solve the problems more effectively as well as learn almost anything you need to about the invocation you are analyzing.</p><p>Try it out and give us your feedback via our <a href="https://thundra-community.slack.com/join/shared_invite/enQtMzA5OTMxNzg1OTUyLTU3MTM2Yjc2M2M1NzNhNGI5ZmZhOGVhMjYxZmIwYTM5ZTdhY2NlZGUyODY3MjlhYThmNzIzNmZhNWI2YzE1Nzg">Slack</a> channel! And, as always, we welcome your ideas and requests and the opportunity to learn about your serverless observability use cases so we can continue to develop Thundra in ways which you find valuable.</p><p>Want to learn more about Thundra’s other features? Visit our <a href="https://www.thundra.io/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=nodejs_manual_instrumentation&amp;utm_content=code_example">website</a> or try our <a href="https://demo.thundra.io/?utm_source=medium&amp;utm_medium=blog&amp;utm_campaign=nodejs_manual_instrumentation&amp;utm_content=code_example">interactive online demo</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9263c7fd5e0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Contemporary Issues in the Design of a New Language]]></title>
            <link>https://medium.com/@ibrahimgrses/contemporary-issues-in-the-design-of-a-new-language-6b64d00a3c51?source=rss-792d45982230------2</link>
            <guid isPermaLink="false">https://medium.com/p/6b64d00a3c51</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[İbrahim Gürses]]></dc:creator>
            <pubDate>Sun, 14 Jan 2018 20:17:06 GMT</pubDate>
            <atom:updated>2018-01-14T20:34:13.507Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>This post is from my student days written for an assignment in Programming Languages Course in Bilkent University.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*ieo23PshuYVjCekz3NjkFA.jpeg" /></figure><p>Although computer programming has reached into a higher level and we as computer scientist have found ways of communication between machines by the help of the abstraction of low level details, people are still searching better ways of designing reliable languages trying to formulate the criteria’s for a perfect high level programming languages. In this process we certainly learn a lot from our mistakes, for example the first American space probe to Venus, reportedly lost because FORTRAN cannot recognise a missing comma in a DO statement.<em>(1)</em> In order to avoid such fatal mistakes designers of a new language always seek ways of catching them before the execution of the program. Also we may want to design a new compact language focusing on a specific purpose and application not for general use, because of these issues people are designing new languages every often even though we have so many great languages like java or C. This post will introduce the formal techniques and criteria’s for a decent reliable language design.</p><p>Lisp was designed by John McCarthy in the 50’s because of the demand of the symbolic processing in the AI applications. As you see the design of a new language always because of needing to program for specific purpose easily. Today computers are used in many applications in many fields not just for science or banking anymore. Programming languages are categorised as what they do best such as array programming, functional programming, logic programming, system programming, and object oriented programming languages. So in order create new languages computer scientists gives guidelines for better language design. Some of the contemporary requirement for a new language can be listed as simplicity, security, performance, efficient object code, object orientation readability efficient object code, bug detection and verification, efficient type checking and garbage collection. Around most of these ideas people found efficient formal methods with attention focused on the need for a language to have a sound theoretical basis such as lambda calculus. For example, Edsger Dijkstra, in a famous 1968 letter published in the Communications of ACM, argued that GOTO statements should be eliminated from all “higher level” programming languages.<em>(2)</em> And as in 2008 programming language theory (PLT) is a well known and active branch of the computer science. Many subfields of PLT emerged in years such as compiler theory, type theory, formal semantics, program transformation, runtime systems, meta programming, domain specific languages.<em>(3)</em> Designer of the language has to decide in each step of definition of the language such as which scoping will the language use or is the language going to be strongly typed or not ? Such decisions have to make according to the programmer’s needs. For example in JavaScript type checking is done dynamically which causes slower programs, since the programs in this language works on a slow environment like internet we will not need fast programs. All the formal rules and axioms of the language try to guarantee minimum human mistake done by the programmer. Since languages are the tools for expressing algorithms and the interface between human and computer, they have to be human friendly. Readability and writability is the most important criteria’s for guaranteeing that. Here is an example code for printing Hello World on the screen in Befunge language which is an esoteric language: <em>(4)</em></p><blockquote><strong>“lroW olleH”&gt;:v ^,_@</strong></blockquote><p>The language is not readable so not writing program with it won’t be reliable, since programmers are humans and they tend to make errors. It is not a practical way to design your language for a bad programmer, but even the best programmer has limits so the language has to provide a way of readable programs. Simplicity is another sect too. If a non-programmer can understand somewhat understand what the code is doing; it will make things easier. For example the code : <strong>prints (“Hello World!);</strong> is so obvious in terms of what it does.</p><p>If you look at the history of the programming languages, you can see that they are evolving like the species. <em>(5)</em> The design process never stops because it is impossible to design the best language which always satisfies the demand of the programmers in every field. I tried to group the design concerns of the languages and tried categorise the languages used today.</p><p>In my opinion, the evolution process will never stop since we need programs for specific purposes. A fish always swims better in the water better than a dog. Since applications of computer science diverse and computers are everywhere the need of writing portable programs for embedded systems. For example you want to design a language for developing applications working on cell phones and you want the programs to require less power from the battery of the cell phones. This may be a great example of the demand of the programmers from the language designers, because today computers are not as big as a room and have a fixed location or have unlimited amount of energy source. Designers of the languages have to advance their criteria’s because of the definition of the computers tend to change. How would you design a programming language for programming a nano robot that travels inside of a human and monitors the vital parameters? Of course all the criteria’s I listed above are crucial again in here, but what kind of advancement can we make? This one of the important contemporary issue of languages face today. The language designer must then be able to synthesise all of these various concepts in such a way as to produce a language which is defined in a uniform way, which has a logical relationship between the syntax and the semantics, which allows an efficiently executing program to be produced, and which permits programmers to conceptualise a solution to a problem in a straightforward manner… The interrelationships among these design criteria are extremely complex, and it appears that it will be some time before a language emerges which can satisfy all of the needs of a broad class of programming applications. So the evolution of the languages and focusing on one purpose in the design is inevitable.</p><p><strong>Reference List:</strong></p><p>(1) Coarse, “Hints on Programming Language Design” <a href="http://www.cs.berkeley.edu/~necula/cs263/handouts/hoarehints.pdf">http://www.cs.berkeley.edu/~necula/cs263/handouts/hoarehints.pdf</a></p><p>(2) Dijkstra, It . W ., “GOTO Statement Considered Harmful,” CACM,</p><p>Vol . 11, No . 3, March 1968, pp. 147–8 .</p><p>(3)”Information On Programming Language Theory”</p><p><a href="http://en.wikipedia.org/wiki/">http://en.wikipedia.org/wiki/</a> Programming_language_theory</p><p>(4)”Befunge Language WebPage”</p><p><a href="http://c2.com/cgi/wiki?BefungeLanguage">http://c2.com/cgi/wiki?BefungeLanguage</a></p><p>(5) “The History of Programming Languages”</p><p><a href="http://oreilly.com/pub/a/oreilly/news/languageposter_0504.html">http://oreilly.com/pub/a/oreilly/news/languageposter_0504.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6b64d00a3c51" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Extract Table Query Statistics on AWS RDS with Percona Toolkit]]></title>
            <link>https://medium.com/@ibrahimgrses/how-to-extract-table-query-statistics-on-aws-rds-with-percona-toolkit-ab0d3b0f835c?source=rss-792d45982230------2</link>
            <guid isPermaLink="false">https://medium.com/p/ab0d3b0f835c</guid>
            <category><![CDATA[table-usage]]></category>
            <category><![CDATA[percona-toolkit]]></category>
            <category><![CDATA[rds]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[query]]></category>
            <dc:creator><![CDATA[İbrahim Gürses]]></dc:creator>
            <pubDate>Thu, 10 Aug 2017 08:35:21 GMT</pubDate>
            <atom:updated>2017-08-10T15:26:28.909Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the easiest way of increasing scalability of your backend system is to adding a caching layer in front of your data layer. Caching should be considered early days in the lifecycle of software projects, but sometimes you may need start your project without any caching layer due to avoid cache inconsistency bugs. Sometimes, teams may postpone solving hard problems into end of the software lifecycle . As Phil Karlton says caching is hard,</p><blockquote><strong><em>There are only two hard things in Computer Science: cache invalidation and naming things.</em></strong></blockquote><p>Since caching is hard, you may need to analyse your DB table query statistics to find the most queried table in your system, so that applying caching to your most active table gives you more benefit in terms of data availability.</p><p>In this, blog post I will talk about what I did to extract query statistics from our MySQL RDS instance on AWS. There are many paid solutions for database query monitoring but they require an agent to install on your DB server machine and they extract metrics via deep packet inspection. However, RDS is a managed DB service and installing an agent on your DB server is impossible. There are solutions out there which uses MySQL <em>PERFORMANCE_SCHEMA </em>to get query statistics, also you can use it by yourself for troubleshooting, but <em>PERFORMANCE_SCHEMA</em> is hard to master and since I am not DB expert, It was hard for me to understand. See <a href="https://dev.mysql.com/doc/refman/5.5/en/performance-schema.html">docs</a> for yourself.</p><p>I found a more simple solution which is<a href="https://www.percona.com/software/database-tools/percona-toolkit"><em> Percona Toolkit</em></a><em>’s </em><a href="https://www.percona.com/doc/percona-toolkit/LATEST/pt-table-usage.html"><em>pt-table-usage</em></a><em> </em>utility. It is open-source and free to use and it was good for my case which is learning which table is more used for a given period of time.</p><p><em>pt-table-usage</em> reads DB log files. The log should be in MySQL’s slow query log format. So we need to enable logging in RDS. Be aware enabling logging on MySQL can reduce performance of your DB and requires a reboot to enable it. So, use it wisely. Here is to tutorial for how to enable it,</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FYKoTiLOvd_Y%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DYKoTiLOvd_Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FYKoTiLOvd_Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/eecc8f6d9a99418ebdf3e15cad7062fb/href">https://medium.com/media/eecc8f6d9a99418ebdf3e15cad7062fb/href</a></iframe><p>After, enabling logging download your log files. I collected 12 hours of logs. Now let’s install <em>Percona Toolkit</em>, we need a linux machine, I used AWS latest Ubuntu AMI.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aca32d44ab42d4fb29b157734b4bb860/href">https://medium.com/media/aca32d44ab42d4fb29b157734b4bb860/href</a></iframe><p>After I tried <em>pg-table-usage</em> without any options, it spitted out id of the query and TableNames used in the query something like this,</p><blockquote>&lt;id&gt; &lt;TableName&gt; &lt;new-line&gt;</blockquote><p>I wrote a golang script which reads lines below and counts Table usages and writes the result to file called output.csv. I like using go instead of bash scripts, because golang compilers gives self sufficient executables for the platform they are compiled. And I can easily write go programs which reads input from STDIN and use them with pipes. I highly recommend learning golang for backend devs at least on basic level. Here is the go code I wrote :</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d30d07f46375fea9250fbb33767746c3/href">https://medium.com/media/d30d07f46375fea9250fbb33767746c3/href</a></iframe><p>Compile the golang code above an start analysis with the command below. When you run the script below, it will do the analysis and write outputs to output.csv file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a8528e9aacc4a12a81fb097f53a5039/href">https://medium.com/media/9a8528e9aacc4a12a81fb097f53a5039/href</a></iframe><p>Once I obtained the csv file, I removed query results for the system tables of MySQL and left with our domain tables. I used <a href="https://plot.ly/">plot.ly</a> to visualise the results. Here the graph I got without the TableNames.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKJcXdJDHf5bc-MYjfEZzw.png" /><figcaption>Query Counts Per Table Graph</figcaption></figure><p>And from the graph we can easily see the most queried tables, first we will implement caching for the top 3 tables. By enabling caching, for the most used table will give us the most benefit in terms of data availability with little development effort.</p><p>In conclusion, I am not a expert on the subject of DB performance monitoring. If you have any recommendations, objections or critics about the post, feel free write comments.</p><p>Links:</p><ul><li><a href="https://dev.mysql.com/doc/refman/5.5/en/performance-schema.html">MySQL :: MySQL 5.5 Reference Manual :: 22 MySQL Performance Schema</a></li><li><a href="https://www.percona.com/doc/percona-toolkit/LATEST/pt-table-usage.html">pt-table-usage</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ab0d3b0f835c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>