<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chirag Khandelwal on Medium]]></title>
        <description><![CDATA[Stories by Chirag Khandelwal on Medium]]></description>
        <link>https://medium.com/@ElectruxR?source=rss-852839018f8a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Q6qfbD1fr4XsUCCd-9v1iA.png</url>
            <title>Stories by Chirag Khandelwal on Medium</title>
            <link>https://medium.com/@ElectruxR?source=rss-852839018f8a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 11:36:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ElectruxR/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[My Workflow: Systems Programming for Linux on Window]]></title>
            <link>https://yc.prosetech.com/my-workflow-for-linux-development-on-windows-1904da475620?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1904da475620</guid>
            <category><![CDATA[virtualization]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Thu, 10 Dec 2020 02:32:14 GMT</pubDate>
            <atom:updated>2021-10-15T04:28:07.317Z</atom:updated>
            <content:encoded><![CDATA[<h3>My Workflow: Systems Programming for Linux on Windows</h3><h4>Simple and efficient C/C++ *nix programming on Windows</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d_PgjGeJdsD1qnZAip0ypg.png" /></figure><p>System software development has always been my passion. It’s so much fun! But with languages like C/C++, it’s not always the easiest to work on Windows — especially when the OS you’re developing for isn’t Windows.</p><p>So, what do I do? Simple. I use VS Code and Virtual Machines! Aaand that’s the end of this article…</p><p>Just kidding. Let me elaborate.</p><h3>The Host OS</h3><p>Honestly, it’s been quite a long time since I last used Windows. However, I recently got myself a new PC (yay!), so I got back to using Windows. As much as I enjoy working on Linux, gaming isn’t its biggest strength. Plus, I wanted a change.</p><p>So, here I am, typing this article because, alas, as much as I am peacefully using Windows, I can’t really do my software development work on it.</p><p>That said, I do use a couple programs to make my life a bit easy — Microsoft <a href="https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-windows?view=powershell-7">PowerShell Core</a> and <a href="https://github.com/microsoft/terminal">Microsoft Windows Terminal</a>.</p><h3>Virtualization</h3><p>Since software development on Windows is a no-no for me, like any sane person I installed VirtualBox.</p><p>Of course you could use Docker or some other software as well if it works for you, but I prefer to use a VirtualBox virtual machine for persistence, ease of use, and most importantly, to have the complete OS available.</p><p>A thing I <em>always</em> do is enable <strong>Bridged Adapter</strong> in VirtualBox VM’s Network Settings and use the generated MAC address to assign a static IP on my router for the machine. That way, the private IP is always constant for the machine and I can access all the services/ports of the machine just like I would for any other device on the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*J8mgOHeCAEutJywFwREpyQ.png" /><figcaption>Setting bridged adapter as the network interface in VirtualBox -&gt; VM Settings</figcaption></figure><h3>The Guest OS</h3><p>Ooh! Time to decide the OS/distro to use for development!</p><p>Honestly, you can choose whatever you like — FreeBSD, OpenBSD, Gentoo, Arch, Ubuntu, and so on. Too many choices to even list here, only dependent on your requirements.</p><p>Historically, I have always been a fan of Gentoo and Arch distributions (I love watching software compile — guess I am weird that way 😛). However, I didn’t want to invest much time setting them up, since this is only a guest OS.</p><p>I went ahead with Ubuntu 20.04. Since I don’t use GUI and would rather like a minimal OS, I opted for Ubuntu’s Server Edition.</p><h3>Host-Guest Interfacing</h3><p>When using VirtualBox and starting a virtual machine, there is a nifty little trick— the drop-down button on the “Start” button allows you to start the Virtual Machine in <em>headless</em> mode. No window for the machine will be presented, but the machine will be running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*Yd0kVC9f5W3YZEU5frbI_g.png" /><figcaption>Starting VM in Headless mode — VirtualBox</figcaption></figure><p>I use SSH to connect from Windows Terminal on my machine to the virtual machine. Using SSH to connect to guest is really easy and efficient. No need to use the (often) slower GUI on virtual machine to open a terminal or, either enable host-guest copy/paste or perform some forbidden dark arts to get data across host/guest.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/995/1*_apdvZ5URNYo4fKEZXFldw.png" /><figcaption>PowerShell on Windows Terminal, connecting to the virtual machine</figcaption></figure><p>As a bonus, since no desktop environment is running, the RAM usage is quite low (as visible in the image)!</p><h3>Code Editor</h3><p>Honestly, I like vim — simple to launch and use. But for anything more than a script file, it’s not practical. I enjoy VS Code’s interface too, and am quite accustomed to it.</p><p>The only issue with using VS Code was that the source code lies on the virtual machine. Notice the “was”? Yep! Turns out there is a (really easy) solution for that too!</p><p>If you check out VS Code’s extensions, there is one from Microsoft called <strong>Remote — SSH: Editing Configuration Files</strong> which allows you to use SSH to use VS Code as a remote editor. And it works just like it would on a local machine!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/1*G6Z5TtJQk_xMl_PjeMF4-A.png" /><figcaption>VS Code connected to VM using the Remote SSH extension</figcaption></figure><p>VS Code also maintains separate extensions for both local and remote so you can pick and choose per your requirements!</p><p>I cannot describe how happy I was when I discovered this feature. It was like all my worries just flew away… yeah that was probably a bit too dramatic, but you get the idea!</p><h3>Conclusion</h3><p>Well, that’s basically my setup for *nix system software development using Windows as my primary OS.</p><p>Another significant benefit I find of using this approach is that I can just leave my PC when I am tired of sitting on a chair, get comfy on my couch/bed, and use my laptop to connect to the virtual machine and continue the work. Pretty smooth, if you ask me!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T6fvt7zIE5wFwuhAGWvjlg.png" /><figcaption>Final Setup — VS Code + Windows Terminal, connected to the VM</figcaption></figure><p>Anyway, that’s it for this article. I recently got into this workflow and really enjoyed it so I thought I should share. Let me know your thoughts and do share your workflow! 😁</p><p>Thanks for reading! Until next time!! ❤️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1904da475620" width="1" height="1" alt=""><hr><p><a href="https://yc.prosetech.com/my-workflow-for-linux-development-on-windows-1904da475620">My Workflow: Systems Programming for Linux on Window</a> was originally published in <a href="https://yc.prosetech.com/">Young Coder</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Writing C++ Modules For Feral]]></title>
            <link>https://levelup.gitconnected.com/writing-c-modules-for-feral-391c30ac7739?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/391c30ac7739</guid>
            <category><![CDATA[interpreters]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[programming-languages]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Tue, 21 Apr 2020 03:20:29 GMT</pubDate>
            <atom:updated>2020-04-21T03:20:29.901Z</atom:updated>
            <content:encoded><![CDATA[<p>Extending the Feral programming language using C++</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f7rwtzF9ijge-f6OQH89ag.png" /><figcaption>Build script for the module created in this article</figcaption></figure><p>In my <a href="https://medium.com/@ElectruxR/the-feral-programming-language-81f87deb58cc">last article</a>, I introduced the <a href="https://github.com/Feral-Lang">Feral</a> programming language. This time around, we’ll create C++ extensions for it! These extensions are building blocks for adding functionality and library support for the language. Usually, a generic task can be implemented in Feral itself, but writing it in C++ is especially useful when the task requires native performance.</p><p>For the purpose of this article, we’ll be implementing a C++ module, containing a Feral variable and function, to demonstrate how to properly utilize the available features to our advantage.</p><p>Any doubts, questions, etc. can be asked either on GitHub or on Feral’s discord server (links at bottom). 🙂</p><h3>Concepts</h3><p>In Feral, a C++ extension works as follows:</p><ol><li>Load the extension at runtime using the mload() function in our Feral script. Internally, this will open the given dynamic library (the module) and call a special function in it called init_&lt;module_name&gt;() (created in the module using the INIT_MODULE(module_name) macro).</li><li>Inject/Insert the required variables and functions into Feral runtime through the init_&lt;module_name&gt;() function.</li><li>The variables and functions are now available to be accessed from the Feral source.</li><li>If there are dynamically allocated variables, they can be deallocated using the optional deinit_&lt;module_name&gt;() function (created using DEINIT_MODULE(module_name) macro). An example for this is the <a href="https://github.com/Feral-Lang/Feral-Curl">Feral-Lang/Feral-Curl</a> module.</li></ol><p>That’s fundamentally it! Easy enough, I think… I hope at least? 😁 🙈</p><h3>Writing Our Module — Creating a Variable</h3><p>For a simple module with explanations and complete source code, check out <strong>ImMaax</strong>’s repository: <a href="https://github.com/ImMaax/Feral-HelloWorld">ImMaax/Feral-HelloWorld</a>.</p><p>wOkay, let’s go for the fun stuff! First of all, we need to create a directory structure for our module. Feral comes with a build system which, when provided with a particular directory structure and a build script (build.fer), will build and install your Feral C++ module for you. Simple as that.</p><p>The directory structure is as follows:</p><pre>&lt;a_module_directory&gt;/<br>    ├─ include/<br>    ├─ src/<br>    └─ build.fer</pre><p>The src/ directory will contain the C++ source files and the include/ directory will contain the .fer import script. Note that neither of them are necessary. We will create our .cpp module source file in the src/ directory. For now, let’s name it learn.cpp (yep, I know — so creative!).</p><p>Now our directory structure looks like this:</p><pre>&lt;a_module_directory&gt;/<br>    ├─ include/<br>    ├─ src/<br>    │   └─ learn.cpp<br>    └─ build.fer</pre><p>Now, we are ready to dive in the (gorgeous) C++ source code.</p><p>Since all the required declarations are present in the feral/VM/VM.hpp header file in the PREFIX_DIR of your Feral installation (more on that <a href="https://feral-lang.github.io/Book/01-install.html#compiler-and-virtual-machine">here</a>), we will have to include that file in our source.</p><p>We also use the INIT_MODULE macro which provides an entry point to our module for Feral. Our source file will look something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c480d1746a04eead67b2a33b26850061/href">https://medium.com/media/c480d1746a04eead67b2a33b26850061/href</a></iframe><p><strong>Note</strong> that the module name <strong>must</strong> be the same as source file name (minus .cpp part), including the alphabet capitalization.</p><p>Let’s create a source local variable — say pi (3.14157).</p><p>Since the variable is source local (available only in the Feral source file which called mload() to load this module), we need to get the currently opened Feral source file and then we can add the variable to it.</p><p>To fetch the current source, we basically get the top element in the source file stack of the Feral VM, and store that element in a var_src_t* type variable.</p><p>This sums up to: var_src_t * src = vm.current_source();.</p><p>To actually insert the variable in source, we have to call the add_native_var function on the src pointer. The entire statement for inserting pi in the source is:</p><pre>src-&gt;add_native_var(&quot;pi&quot;, make_all&lt;<em>var_flt_t</em>&gt;(3.14157, src_id, idx));</pre><p>Let’s break this down.</p><p>From the innermost side, we see make_all&lt;var_flt_t&gt;. In essence, var_flt_t is a Feral type for storing floating point values. make_all is a function that allows us to initialize the type (here var_flt_t) with required values.</p><p>The arguments for var_flt_t are:</p><pre>1. float: the floating point value to be created<br>2. size_t: the current source ID<br>3. size_t: the current source location index</pre><pre>For the first argument, we enter the pi&#39;s value - 3.14157.<br>src_id and idx are provided to us through the INIT_MODULE macro itself.</pre><p>The src_id and idx variables are crucial as, on occurrence of an error, they are used to show the source file, line, and column where it occurred.</p><p>The first argument to src-&gt;add_native_var() is the name with which our variable will be available in Feral (here pi).</p><p>Well, that’s it! We created our first variable for Feral in our extension! The entire source code should look like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/36a311a73d7f064983c7e908b97ba6e9/href">https://medium.com/media/36a311a73d7f064983c7e908b97ba6e9/href</a></iframe><p>Now, what’s left to do is create our build script (build.fer) and install this module for us to test. The build script is as follows:</p><pre>let <em>sys</em> = import(&#39;std/sys&#39;);<br>let <em>builder</em> = import(&#39;std/builder&#39;);</pre><pre>let <em>build</em> = builder.new().make_dll();<br>build.add_src(&#39;src/learn.cpp&#39;);</pre><pre>sys.exit(build.perform(&#39;learn&#39;));</pre><p>The only two things we need to note in this script for now are: build.add_src(&#39;src/learn.cpp&#39;); and sys.exit(build.perform(&#39;learn&#39;));.</p><p>These are our source file location and the build name respectively. Just remember to name these according to your file location and module name.</p><p>After that, we will install our module using the feral install command. But first, make sure that Feral’s module directories are initialized using feral init command. For me, the output of feral install is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*Zow6PAvCl1d7rbXoa8Wi3Q.png" /><figcaption>feral install</figcaption></figure><p>Yay! Time to test this out!</p><p>Just create a script — say test.fer, import the io module and mload() our custom module in that file, and use the io.println() function to see if we can print the value pi. The code for that is as follows:</p><pre>let io = import(&#39;std/io&#39;);<br>mload(&#39;learn&#39;);</pre><pre>io.println(&#39;Value of pi is: &#39;, pi);</pre><p>You’ll get the following output:</p><pre>Value of pi is: 3.14157000000000019568</pre><p>Yay!! We made our first C++ extension!!</p><p>But, it isn’t complete yet. Notice the mload() instead of import()? Yea, we don’t want that. The reason being that the symbols in it cannot be stored in a variable (like io), which means it is unusable outside this script.</p><p>So, we want to wrap this mload() call in a Feral script which we will import as required, instead of using mload() again and again.</p><p>To do that, create a Feral script — say learn.fer in the include/ directory, and just write the mload() function call in that script:</p><pre>mload(&#39;learn&#39;);</pre><p>Now, do a feral install again and boom! we are done with creating our simple module!!</p><p>To use it, instead of calling mload(), we will call import() and store it in a variable which will contain our pivariable. Our final test script will look like this:</p><pre>let io = import(&#39;std/io&#39;);<br>let learn = import(&#39;learn&#39;);</pre><pre>io.println(learn.pi);</pre><p>Woohooo! Our first C++ extension for Feral! Isn’t it awesome?! 🤩😍</p><p>Time to create a C++ function for Feral!</p><h3>Writing Our Module — Creating a Function</h3><p>A Feral function’s C++ signature is specific. We must use that signature to properly create a Feral function. The signature is:</p><pre><em>var_base_t</em> * func(<em>vm_state_t</em> &amp;, const <em>fn_data_t</em> &amp;);</pre><p>The function name (here func) can be whatever you want. The function must have 2 arguments:</p><pre>1. vm_state_t&amp;: virtual machine state - provides access to the VM<br>2. const fn_data_t&amp;: the function call information - mainly arguments, keyword arguments, src_id, and idx</pre><p>And, the function must return a Feral variable object pointer (var_base_t*).</p><p>Well, let’s create a function that returns length of a string!</p><p>We know, in C++, the length of string is returned by the std::string::size() member function. We can leverage that. And of course, Feral has a wrapper for C++’s string type. You guessed it — var_str_t.</p><p>We will create a function len() which will take the string as parameter, and return the length of that string. The code for that is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/125176cafbfe417d3850f4d6e671d2f4/href">https://medium.com/media/125176cafbfe417d3850f4d6e671d2f4/href</a></iframe><p>… Yea, we should break it down.</p><p>The first line is of course, the function signature we discussed about — the function name here is len. Inside the function body, we fetch the src_file object of the source file currently being executed — this is done to properly produce errors. Then, we check if the type of our first argument (fd.args[1]) is string (VT_STR) or not. If it is not a string, we use the vm.fail() function to show an error on the position of our first argument (fd.args[1]-&gt;idx()) and return nullptr.</p><p>Note that the first argument is not fd.args[0] because that is reserved. We’ll get to that later. Also, when we return nullptr it actually means that the function has failed and Feral will stop executing after that.</p><p>Finally, we typecast our base class object fd.args[1] to var_str_t type using the STR() macro, then we get() the wrapped std::string object and use the size() member function to fetch the length of it, after which, we wrap that value to a var_int_t and return it. That’s it!</p><p>You must have noticed that this time we used make&lt;&gt;() function instead of the previous make_all&lt;&gt;() that we used. The difference between them is that we do not set the src_id and idx when using the make function which is what we want here because after the function call, Feral sets that value by itself.</p><p>There we have it! Our own Feral function! However, we also need to let Feral itself know about this function’s existence. For that, we’ll add it to Feral’s currently executing source file (similar to how we added our variable before).</p><p>We will, therefore, use the add_native_fn() member function of our src variable in the INIT_MODULE block.</p><p>The arguments for that function are:</p><pre>1. string: the name for function which will be visible in Feral<br>2. nativefnptr_t: the function we created in the C++ source<br>3. int (optional): number of arguments required by the function<br>4. bool: does the function use variadic arguments</pre><p>Since we want our len function to be called len in Feral, which takes 1 argument and is not variadic, the function call will be as follows:</p><pre>src-&gt;add_native_fn(&quot;len&quot;, len, 1);</pre><p>Yep! We’re done!! We can try our function now! 😁</p><p>Let’s feral install again to update our module and write the following in test.fer:</p><pre>let <em>io</em> = import(&#39;std/io&#39;);<br>let <em>learn</em> = import(&#39;learn&#39;);</pre><pre>io.println(learn.len(&quot;some string&quot;));</pre><p>When we run it, we’ll see the output 11 which is the length of some string. Yay! it worked!! 🤩</p><p>Although, wouldn’t it be better if we could do something like… &quot;some string&quot;.len()? Let’s do that!</p><p>To create a member function (or correctly, type bound function) we need to bind the function to a specific type (here VT_STR) and change a little bit of our function body.</p><p>For the function body, remember we had a reserved fd.args[0]? Well, it is actually the variable that contains the object that called this function. In other words, if we do &quot;some string&quot;.len(), fd.args[0] will contain &quot;some string&quot; (similarly, when we did learn.len() learn object was contained by it).</p><p>Also, since this function will be bound to the string type, we will not have to check if the argument is a string like we did before!</p><p>Performing those changes, our new function becomes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/87fed3b80ba44c477428b609d217f1c7/href">https://medium.com/media/87fed3b80ba44c477428b609d217f1c7/href</a></iframe><p>Note that we also removed src_file since it was no longer required.</p><p>This looks small and cute, doesn’t it! 😍</p><p>Finally, updating the function declaration in INIT_MODULE, we replace the src-&gt;add_native_fn() call with vm.add_native_typefn(). The arguments for this function are:</p><pre>1. int: type to bind the function to<br>2. string: the name for function which will be visible in Feral<br>3. nativefnptr_t: the function we created in the C++ source<br>4. int: number of arguments required by the function<br>5. size_t: the current source ID<br>6. size_t: the current source location index</pre><p>Since this is a going to be a member function of string type, we no longer have to provide any argument to it. Therefore, the argument count will now be 0 instead of the previous 1. Therefore, the final function call becomes:</p><pre>vm.add_native_typefn(VT_STR, &quot;len&quot;, len, 0, src_id, idx);</pre><p>Now, use feral install to install updated module, and change the test.fer to use &quot;some string&quot;.len() instead of learn.len(&quot;some string&quot;).</p><p>The code becomes:</p><pre>let <em>io</em> = import(&#39;std/io&#39;);<br>let <em>learn</em> = import(&#39;learn&#39;);</pre><pre>io.println(&quot;some string&quot;.len());</pre><p>Note that the learn. prefix is not required anymore.</p><p>Execute the code and we’ll see the output 11 which is the length of some string. Perfect!!</p><p>The entire source code for our C++ module is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/172305fd6098917918f2753f05c3d249/href">https://medium.com/media/172305fd6098917918f2753f05c3d249/href</a></iframe><h3>Conclusion</h3><p>This was a basic article/tutorial on how to create C++ extensions for Feral. I hope it was informative and fascinating. 😁</p><p>If you want more examples look in the <a href="https://github.com/Feral-Lang/Feral-Std">Feral-Lang/Feral-Std</a> repository (the standard library for Feral). The relevant links are given in the section below.</p><p>As always, all questions, ideas, suggestions, and thoughts are pleasantly welcome and greatly appreciated.</p><p>Thanks a lot for reading, and have a great day. Until next time! ❤️</p><h3>Links</h3><p>Feral Discord Server: <a href="https://discord.gg/zMAjSXn">https://discord.gg/zMAjSXn</a></p><p>Feral Lang (Organization URL): <a href="https://github.com/Feral-Lang">https://github.com/Feral-Lang</a></p><p>Feral Compiler/VM: <a href="https://github.com/Feral-Lang/Feral">https://github.com/Feral-Lang/Feral</a></p><p>Feral Std (Standard Library): <a href="https://github.com/Feral-Lang/Feral-Std">https://github.com/Feral-Lang/Feral-Std</a></p><p>Feral Book (WIP): <a href="https://feral-lang.github.io/Book">https://feral-lang.github.io/Book</a> (source: <a href="https://github.com/Feral-Lang/Book">https://github.com/Feral-Lang/Book</a>)</p><p>Feral HelloWorld (by ImMaax): <a href="https://github.com/ImMaax/Feral-HelloWorld">https://github.com/ImMaax/Feral-HelloWorld</a></p><p>Previous Article: <a href="https://medium.com/@ElectruxR/the-feral-programming-language-81f87deb58cc">https://medium.com/@ElectruxR/the-feral-programming-language-81f87deb58cc</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=391c30ac7739" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/writing-c-modules-for-feral-391c30ac7739">Writing C++ Modules For Feral</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing the Feral Programming Language]]></title>
            <link>https://yc.prosetech.com/the-feral-programming-language-81f87deb58cc?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/81f87deb58cc</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[interpreters]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[c]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Sun, 05 Apr 2020 14:00:56 GMT</pubDate>
            <atom:updated>2020-04-10T01:38:05.688Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://medium.com/young-coder/ethereal/home">Creating a Programming Language</a></h4><h4>My next step from the Ethereal programming language</h4><p>Recently — a couple months ago in fact — I started working on the <a href="https://github.com/Feral-Lang/Feral">Feral</a> programming language. Now, if you have read my previous articles, right now you’re probably like: “Dude! Another language?! What’s wrong with you!” And come to think of it, you’re probably right. I seem to be obsessed with programming languages, but that’s perhaps for another article. Right now, I can’t wait to share about the Feral Programing Language!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*EvWxCJS8yRffxO7krPSWDA.png" /><figcaption>Sample Feral Code — Bubble Sort! (the best sorting algorithm!! (just kidding))</figcaption></figure><h3>Some Background</h3><p>I started working on Feral about 2-3 months ago. I really enjoy creating languages, so I thought why not! This time around though, I focused a lot more on the syntax of the language than the last time (<a href="https://medium.com/young-coder/so-i-created-a-programming-language-4d9c11038d22">Ethereal</a>). I wanted to make a language that’s not just pleasant to use, but also minimalistic, could be picked up in like, a day, and didn’t sacrifice usability or readability while not being overly verbose (<em>cough</em> Java <em>cough</em>). That was (and still is) the foundation of Feral — a language which is fun to script in!</p><p>I also wanted to play around with some ideas like first-class support for modules and functions — i.e., modules and functions can be passed around in the form of variables! Now, I know that this is nothing revolutionary, but come on! It’s pretty cool and fascinating! 😁</p><p>Armed with all the ideas, motivation, and well, a borderline obsession towards creating languages, I started working on Feral — the minimalistic, interpreted, procedural programming language.</p><p>Therefore, off I went to my trusty whiteboard and code editor and started scribbling down the syntax of the language and sample programs, all the while, making it better and something that I felt would be really enjoyable to use. That’s when Feral was born! By the way, I gave it the name <strong>Feral</strong> because:</p><ol><li>It’s a lovely word!</li><li>It’s wild — Fast, and powerful!</li></ol><h3>The Language</h3><p>Remember, I said that Feral is minimalistic? Well, I <strong>meant</strong> it! Feral does not contain any special syntax for structures, enumerations, maps/dictionaries, or even vectors/lists. Hell, there is no special syntax for even importing other source files.</p><p>Do note that I said it didn’t contain special syntax for those things. I did <strong>not</strong> say that it doesn’t <em>support</em> them. Feral does support all of those concepts. However, instead of relying on special syntax on the language side, it provides functions with which the same functionality is created while having far less core compiler codebase.</p><p>Feral contains an import() function which allows the user to import other Feral sources, instead of relying on an import keyword that most other languages have. Similarly, structures are created using lang.struct() function and enumerations are created using lang.enum() function.</p><p>Feral also has first-class support for modules and functions! And not just that, even structures are first-class citizens in Feral (in a way, we’ll see how, below).</p><p>In Feral, the modules (cool way of saying “other feral sources”) are loaded as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*N07cdzTLU46pLzkciUuPjw.png" /><figcaption>Importing the standard io module in Feral</figcaption></figure><p>Basically, <strong>io</strong> is now a variable which can be passed to functions, returned from functions, reassigned to another value, basically, it has all the functionality of any other variable!</p><p>Similarly, to create a structure (<strong>not </strong>an instance/object of structure), the lang.struct() function is used as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*8wjO_sJ8W7tbaMIwD15ULQ.png" /><figcaption>Importing ‘lang’ module and calling struct() function in it — creating a new structure</figcaption></figure><p>Finally, just call my_struct as a function to create an instance of that struct. Fascinating, right?!</p><p>Even functions themselves are Feral <em>values</em> which can be assigned to variables and passed around to other functions (callbacks). So, say you want to make a sort function and pass the sorting comparison mechanism separately, as a function itself? Check! (Although, for the love of everything holy, please don’t overuse callbacks, ever!)</p><p>In Feral, as with many other interpreted languages, one can write functions and types in C++ (host language) as C++ runtime loaded modules, or as Feral functions themselves. The simple thought for deciding which one to use is to only go the C++ route if the function requires high performance, otherwise Feral functions will satisfy most of the cases.</p><p>Currently, there is also a Curl module under progress. For example, to download a file using Feral (and its Curl module), the code goes as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*t18IWwIlQ8dDcTmXwSVexg.png" /><figcaption>Feral code for using Curl to download a file — It’s so adorable and cute isn’t it?! 🥺❤️</figcaption></figure><p>I am working on the documentation of the language right now (5 chapters are complete) and will start uploading it soon enough.</p><h3>Syntax Highlighting</h3><p>I don’t know about you, but I most certainly can’t live without it. I need my language syntax to be highlighted by colors after all, otherwise it would look too dull and boring!!</p><p>Keeping that in mind, I have also created simple syntax highlighting extensions for Visual Studio Code and Vim (links at the bottom).</p><h3>Technical Details</h3><p>This is an article of a programming language, what fun would it be if there were no technical details!</p><p>Since Feral is an interpreted language, it required a host language which would be, ideally, natively compiled. For me, <strong>C++(11)</strong> was a no brainer here since it is natively compiled, outrageously fast, and I love to use it (I dare you to sway me to another language!).</p><p>As a matter of fact, the only issue I have with writing C++ code is regarding the build system — I spent a lot of time trying to understand how to get my work done with CMakeLists. I don’t hate CMake, but it can get annoying to get its configuration right. Alas, it does work pretty well and is ubiquitously available, so CMake it was!</p><p>For the language implementation itself, the Lexer was nothing special, the parser on the other hand, I am quite happy with what it has become. This time around (unlike Ethereal), I actually made a <strong>Recursive-Descent Parser</strong>, and the code looks so beautiful and clean! I definitely recommend it if it works for your language syntax.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*4b9xEkQP2HziXxCLhi7JtA.png" /><figcaption>A small part of Feral’s expression parsing — Recursive Descent Parser</figcaption></figure><p>No, I don’t use tools like yacc, or bison — I like being in total control of the codebase, plus for me, yacc/bison would have taken a lot more time to understand and then put into use.</p><p>The compiler/VM uses libgmp and libmpfr library for working with arbitrary sized numbers. This totally eliminates the need to package a separate library for working with big numbers. Aside from system libraries, these are the only external libraries required for installing and using the Feral programming language.</p><p>But my oh my, the real fun was in writing the Virtual Machine. Since there was always the goal in mind of the language being minimalistic, the number of VM instructions would have to be low too. As of now, Feral contains a total of <strong>22</strong> VM instructions through which any Feral program is executed. I think that’s sufficiently low.</p><p>The VM itself is stack-based, therefore each instruction is quite simple — an operation, and an operand (which is optional).</p><p>Currently, VM consists of the following instructions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HSYDjDDfxfeDaUaM3YROag.png" /><figcaption>Feral VM Opcodes</figcaption></figure><p>As of yet, to be honest, the VM is most certainly not quite optimized. I am working on that, but it’s definitely not agonizingly slow — by a long shot!</p><h3>Conclusion</h3><p>Well, this is a very short introduction, with the main things I feel to be interesting about Feral. As the development continues, the language is becoming more and more stable, and I am incredibly excited about working with it and using it.</p><p>Hope you all find the article and language interesting/exciting! Comment your thoughts on them and do check out Feral. Feedback is always welcome and most appreciated! But that’s it for today.</p><p>Until next time everyone! ❤️❤️</p><h3>Links:</h3><p>Feral Language Organization: <a href="https://github.com/Feral-Lang">https://github.com/Feral-Lang</a></p><p>Feral Compiler/VM: <a href="https://github.com/Feral-Lang/Feral">https://github.com/Feral-Lang/Feral</a></p><p>Ethereal Language: <a href="https://github.com/Electrux/Ethereal">https://github.com/Electrux/Ethereal</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81f87deb58cc" width="1" height="1" alt=""><hr><p><a href="https://yc.prosetech.com/the-feral-programming-language-81f87deb58cc">Introducing the Feral Programming Language</a> was originally published in <a href="https://yc.prosetech.com/">Young Coder</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[…You Should Develop A Programming Language]]></title>
            <link>https://medium.com/@ElectruxR/you-should-develop-a-programming-language-3ca6cb6a9b4?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ca6cb6a9b4</guid>
            <category><![CDATA[interpreters]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[language-development]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Thu, 19 Dec 2019 20:50:07 GMT</pubDate>
            <atom:updated>2019-12-19T20:50:07.724Z</atom:updated>
            <content:encoded><![CDATA[<p>No, I am not joking. It can be quite an interesting experience. Let me tell you why.</p><h3>Introduction</h3><p>Personally, I really enjoy developing programming languages — that’s my hobby. Currently, I am developing my interpreted programming language: <a href="https://github.com/Electrux/Ethereal">Ethereal</a>. Do I think it will be the next big thing? Nope. A breakthrough? Nada. A juggernaut that causes the end of world? Well, I certainly hope not!</p><p>Then why? Why must I, or for that matter, anyone, spend so much time developing something that shall, most likely, not be used by anyone?</p><p>The same reason as to why we attempt implementation of data structures when we learn to code.</p><h3>It Teaches A Lot</h3><p>Language design and development provides a huge amount of knowledge — not just about programming, but all the way from design and architecting, to the intricacies of programming — data structures, optimization and internal working of computers themselves.</p><p>The first question that might come to mind is, why should I care about the internal working of computers? I work on web development after all!</p><p>My friend, the reason you should care about it is that it will allow you to develop <strong>better</strong> software, period. You’ll be able to understand why your web application might be running slow, how can you make it efficient, and how to <strong>optimize</strong> it. It doesn’t matter if you are working on Android Development, Desktop Applications, IoT, or even Data Science. The knowledge you gain will be invaluable nonetheless.</p><p>Not just this, but it can also help improve the knowledge of the language itself. When developing a programming language, you will learn a lot about the language you develop it in. And you are not even limited to a specific language. You can develop a language even in Python and JavaScript!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KN0r63vdx5aDm2IpjuGIMg.jpeg" /><figcaption>So many languages in which languages can be written</figcaption></figure><h3>It Is Absolutely Ecstatic</h3><p>Imagine being writing some code. Feels great to develop something new right. Now imagine that the language you are writing the code in is yours!<br>I don’t know about you, but to me, it sure as hell is exciting.</p><p>The sheer joy of writing code and realizing that it is being written in your own language is, well, immeasurable. Developing something new is fun, and developing something new in your own language makes the fun grow <strong>exponentially</strong>. Of course, you can go one step ahead and develop your own Operating System to work on, but that is totally out of this article’s scope.</p><p>Working on the design and implementation of a language feels equally amazing. Looking at the language taking shape according to your thoughts, and generating expected output when working on the implementation is really satisfying.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*LN_Zzz6uy2m6UUX_Tt663Q.png" /><figcaption>Writing code in your language</figcaption></figure><h3>Ugh… Bragging Rights, Anyone?</h3><p>Let me be clear, this is not a reason for developing your language. Well, not the primary or sole reason anyway! But come on, no one can deny that it does feel and look cool! And most certainly, people will raise their eyebrows if you tell them that you developed a language. Not because it is an amazing feat to do so (unless we are talking about the actual juggernaut compilers/interpreters — LLVM, CPython, etc), but because it is simply rare and often nebulous, even to software developers.</p><p>It can even give your <strong>resumé</strong> a sizable boost. For a person reviewing multiple resumés, it is quite catchy to notice one which has a programming language in the <strong>list of projects</strong> and not just in the <strong>known languages</strong> section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*nsERo4QCsw3AErPQS74fEg.jpeg" /><figcaption>Yep, I developed a Programming Language, bow before me!!</figcaption></figure><h3>Great University Project</h3><p>We have all seen, and grown tired of, the traditional university projects with changes in UI and may be functionality. Obviously, if you have a good idea falling in that category, more power to you. But more likely that not, having a Programming Language as a university project is relatively unique and will certainly impress your faculties/mentors — so long as you work on it properly!</p><p>Personally, my undergrad university project was a programming language (looking back, pretty trash!) that I made. And it was still considered a really good project — not because of some amazing implementation (or lack thereof), but simply due to the uniqueness of the project and how it could be used. Let’s just say, it will certainly get you quite some bewildered and star-struck looks!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dKeUZWRDwdBzSbRiVr-YhQ.jpeg" /></figure><h3>Conclusion</h3><p>When I started programming, I never thought I’d be working on developing languages themselves! Frankly, a lot of it still seems obscure and nebulous to me, but I continue to learn and enjoy the experience.</p><p>One thing is for sure — Language development has me hooked. I have developed programming languages and I don’t see myself stopping anytime soon.</p><p>I just hope that at least some of you find it equally interesting, fascinating, and share my love for it.</p><p>Until next time! ❤️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ca6cb6a9b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How I Wrote A Lexer]]></title>
            <link>https://yc.prosetech.com/how-i-wrote-a-lexer-39f4f79d2980?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/39f4f79d2980</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[interpreters]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[lexer]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Fri, 04 Oct 2019 03:23:17 GMT</pubDate>
            <atom:updated>2019-10-04T03:23:17.024Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://medium.com/young-coder/ethereal/home">Creating a Programming Language</a></h4><h4>My experience designing and writing a Lexical Analyzer</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1op5__sWodY4R17ZY7X1Hw.jpeg" /><figcaption>Image by <a href="https://pixabay.com/users/wilhei-883152/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=705667">Willi Heidelbach</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=705667">Pixabay</a></figcaption></figure><p>In previous articles, I talked about my language (<a href="https://github.com/Electrux/Ethereal">Ethereal</a>) , <a href="https://medium.com/young-coder/so-i-created-a-programming-language-4d9c11038d22">my journey with it</a>, and <a href="https://medium.com/young-coder/architecting-the-ethereal-language-ad0b84bb04e3?source=friends_link&amp;sk=5336f28eead14f0e638e35632b814b91">how I architected it</a>. In this article, I want to share how I developed the first component of the language —the lexical analyzer.</p><p>If you have any interest in developing a programming language, you’ve probably looked at lexers and how to develop them. But even if you haven’t, there are more than enough examples and tutorials out there. So, I am going to focus on <em>my experience</em> writing a lexer. I’ll explain the pitfalls I faced, the design decisions I made, and the techniques I used.</p><p>But first of all, let’s understand what are lexers and what do they do.</p><h3>Understanding lexers</h3><p>A lexical analyzer — more commonly referred to as <em>lexer </em>— is a software component that takes a string and breaks it down into smaller units that are understandable by a language. These smaller units are called <em>lexical tokens</em> or <em>lexemes</em>. In other words, you can think of a lexer as a black box that takes a sentence as input and breaks it into smaller units —essentially, words.</p><p>A lexer, however, does more than that. Often it will also check if the broken-down units actually mean something for the programming language. It’s kind of like the spell checker we use in our word processing software. Do note that the lexer will <em>not</em> check the grammatical syntax of the language. (In other words, it doesn’t know if you’re using the right tokens in the wrong combination.)</p><p>Let’s look at an example of a lexer in action.</p><p>The input is this tiny piece of code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/212/1*SGfmhw3wyLmgGHaiF3fsAA.png" /><figcaption>Input to the lexer</figcaption></figure><p>When you feed this code into the lexer I made, you get the following output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/716/1*ZPkOp-ygKpZJ3NyPdaBcmQ.png" /><figcaption>Lexical output — tokens</figcaption></figure><p>In this output, the IDs are simply the indices for each token. The type identifies whether the token is an identifier (IDEN), number (INT), operator (+, +=), separator (;, ()), and so on. Next is the line number with a column number in brackets [], and finally the actual data for dynamic types (identifiers, numbers).</p><p>In terms of a programming language, the lexer’s job is to break the source code down into smaller pieces — lexemes — so that they may be easily used in the next stage by the compiler/interpreter.</p><h3>My first lexer implementation — The bad one</h3><p>When I first started developing the lexical analyzer for my language, being a CS undergrad, my thoughts immediately raced towards <a href="https://www.freecodecamp.org/news/state-machines-basics-of-computer-science-d42855debc66/">Finite State Machines</a>. And since I had researched building lexers, regular expressions also came to mind. Frankly though, I dislike writing regular expressions (just like any sane person!). They are complex to write and understand, and from the perspective of computers, require rather hefty computation.</p><p>So, I started writing a lexer using the concepts of Finite State Machines and ended up with something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cUx3ZC1h0ZMOIAo2aW46MA.png" /><figcaption>FSM — States of Lexer</figcaption></figure><p>Using this model, I iterated through each character of the source code. I used current and previous variables (type and prevtype) to keep track of the type of data being tokenized. Here’s the code that did the job:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*TcF1w36nTPt0lW37pxcWnw.png" /><figcaption>FSM — Loop to find out type of string based on the current and previous character type</figcaption></figure><p>This loop identifies the type of each data string by moving to a different <em>state. </em>The new state is based on the previous state. For example, if the loop encounters multiple digits followed by a dot (.), prevtype would be NUM, and type would become FLOAT. (This makes sense, because the dot operator after one or more digits signifies a floating point value.)</p><p>To be honest, when I first made this, I was proud of myself. I felt accomplished.</p><p>Well… for a little while at least.</p><p>Soon enough, I realized this was a very trivial way to go about things. My code would not be able to handle complex tokens easily (without fiddling at least). As a matter of fact, it couldn’t even handle parsing the usual identifier sequences (underscore, a-z, A-Z, 0–9) as it didn’t know that if underscore is the first character, the string should be categorized as an operator (invalid or otherwise) or the beginning of an identifier. My pride came crashing down.</p><p>Overall, this implementation was a total bust. It could parse simple tokens no problem, but it would be a nightmare to make it parse even a slightly more complex sequence of characters.</p><p>But — as is often the case — you learn more from making big mistakes than getting your code right the first time.</p><h3>The moment of enlightenment</h3><p>One fine morning, after some thinking and reading through a couple of other language implementations, it hit me!</p><p>I had been stuck on Finite State Machines and regular expressions, but at the end of the day, a lexer was just sequentially going through the characters and categorizing them into groups every time it finds a break point (an invalid character, space, operator, etc). With that thought, I started implementing the lexer using simple loops and switch cases to identify the token types.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LBJ-pm2fCVcEw2cIVkdVAw.png" /><figcaption>A small part of the implementation of my lexer</figcaption></figure><p>For example, if the first character in loop is an operator, the get_operator function is called. Then, the code tests several cases (shown above) to see if the operator makes sense. If not, it errors out and returns FAILED.</p><p>This implementation technique improved the code’s readability and extensibility quite a lot.</p><p>Each line of code is looped through and tokenized like this. Even the code for tokenizing an identifier is very small and easily understandable:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*FUVTDgJWtPCdMIvSSE3iew.png" /><figcaption>Tokenizing an identifier</figcaption></figure><p>Once the identifier string is returned, it’s compared with the set of language keywords. This allows the lexer to decide if the identifier is a keyword. If it turns out to be a keyword, the lexer sets the type accordingly and the loop moves on.</p><p>And that is basically how I implemented the lexer for my language. It probably has some issues and edge cases where it will fail, but I have yet to find them.</p><p>To be honest, depending on language syntax, the lexer is usually the easiest component to code. The lexer I wrote using the technique above is about 600 lines in the source file. The entire source of my lexer is available <a href="https://github.com/Electrux/Ethereal/blob/master/src/FE/Lexer.cpp">here</a>.</p><h3>Conclusion</h3><p>At the end of the day, there are a plethora of ways one can write a lexer. Some techniques are easier to implement while others provide great extensibility. Of course, the approach you use also depends on the syntax of your language.</p><p>For those of you interested in writing a language, or even just a lexer, I hope this article is helpful. Feel free to check out <a href="https://github.com/Electrux/Ethereal/blob/master/src/FE/Lexer.cpp">my lexer</a> and learn from it, or just tell me what I did wrong!</p><p>Anyway, thank you for reading! Feedback is always appreciated. I’ll be writing about building the parser next, which is, in my experience, much more sophisticated than lexer (and equally exciting to write).</p><p>Until next time! ❤️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39f4f79d2980" width="1" height="1" alt=""><hr><p><a href="https://yc.prosetech.com/how-i-wrote-a-lexer-39f4f79d2980">How I Wrote A Lexer</a> was originally published in <a href="https://yc.prosetech.com/">Young Coder</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Architecting the Ethereal Language]]></title>
            <link>https://yc.prosetech.com/architecting-the-ethereal-language-ad0b84bb04e3?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad0b84bb04e3</guid>
            <category><![CDATA[interpreters]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cpp]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[compilers]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Tue, 10 Sep 2019 16:17:24 GMT</pubDate>
            <atom:updated>2019-10-03T20:45:17.414Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://medium.com/young-coder/ethereal/home">Creating a Programming Language</a></h4><h4>The decisions I made when I designed my own programming language</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A3KmmMzZhSpgxcv8u_Zyaw.jpeg" /><figcaption>Adapted from <a href="https://pixabay.com/photos/louvre-pyramid-paris-tourism-102840/">Pixabay</a></figcaption></figure><p>Recently, I <a href="https://medium.com/young-coder/so-i-created-a-programming-language-4d9c11038d22">wrote an article</a> with an overview of creating a programming language from my perspective. My language is Ethereal — an experimental work-in-progress that you can <a href="https://github.com/Electrux/Ethereal">check out on GitHub</a>.</p><p>What is it for? Well, my idea is to use it for building scripts. But since Ethereal is a general purpose language, it can be used for whatever you want. And even if I don’t write anything more than a test script, I’ve learned far more on this project about the internal details of how languages work than I would playing around with someone else’s mature platform.</p><p>How did I make the key design decisions about how Ethereal should work?Let’s dive deeper.</p><h3>Introduction</h3><p>In this article, I’ll be writing about the features of Ethereal, their design, the choices I made, and how I went about implementing them. I will talk more about individual components in later articles, but this one will revolve around the overall language design and architecture.</p><p>I will say that I am by no means an incredible language designer/architect. But I do hope that this article provides some insight to people who want to create their own programming languages, or are just curious about them. (If that’s you, great! If that’s not you — why not reconsider?)</p><h3>Language design</h3><p>When I envisioned creating a programming language, I definitely did not expect it to be the new cutting-edge, ground-breaking, sky-encompassing language that would shake the very foundations of the world. Far from it. Originally, I started designing and developing the language simply as a hobby. Most importantly, I wanted to learn from it how programming languages work, how everything falls into place, and what it takes to actually write one.</p><blockquote>It turns out that developing a programming language is not that hard, as long as you are not creating your own version of C++ or Haskell (in which case, tread <strong><em>very carefully</em></strong><em>)</em>.</blockquote><p>Fundamentally, I wanted my language to have some core features:</p><ol><li><strong>Extensibility</strong>. I didn’t want my language to be limited just to some selected instructions. I wanted it to be easily extended depending on the task.</li><li><strong>Decent Performance</strong>. Let’s not kid ourselves, an interpreted language (like Ethereal) will not have nearly as good performance as a compiled language. But still, I didn’t want my language to be sluggish. I want it to be fast enough to work on the task I want, easily. Also, it would be great to have a way to delegate some high performance tasks to a compiled language and integrate that alongside the Ethereal code.</li><li><strong>Turing Completeness</strong>. For those of you who don’t know, this means that any problem which is solvable can be solved using this language. Yes, it may be tremendously hard to do so, but it should be possible nonetheless. This would require the language to have the basic constructs for flow control like conditionals, loops, and recursion.</li><li><strong>Ease of Use. </strong>I wanted a language which can be easily used, hassle free. I didn’t want to invent some exotic syntax just for this language, nor did I expect others to learn it if I did.</li><li><strong>Modularity</strong>. I wanted the language to be modular. I shouldn’t need to write all the code in a single source file. That’s inefficient and promotes needless code duplication.</li><li><strong>Basic Stuff. </strong>All the basic constructs a language usually has — structs, functions, vectors, and maps — should be present. Vectors and Maps should also allow the use of the [] operator.</li></ol><p>With these requirements, I started writing small sample programs with a variety of hypothetical syntax to decide which syntax would be good. I chose the ones that I found pleasing to the eye and amalgamated them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*NiKPHdhuk14_Ex5aD10kZA.png" /><figcaption>Sample — simple program for displaying time since epoch, and time difference in microseconds</figcaption></figure><p>Then, I started writing out more sample programs in that amalgamated syntax and saw the limitations and annoyances in it. So I refined it. When I was satisfied with the syntax, I started working on the technical architecture of the language.</p><h3>The host language</h3><p>Now I had the requirements and the syntax of my planned language. It was time to fill in the implementation that would actually make the syntax work on a computer.</p><p>For writing the language, I chose C++ for two reasons:</p><ol><li>I wanted a fast compiled language. I didn’t choose C, because I didn’t want to lose some abstractions that C++ provides — vectors, strings, maps to name a few. In other words, I wanted the C++ Standard Template Library (STL). It just makes my life so much easier and lets me focus on the work that I’m doing instead of having to implement these constructs from the ground up — and I wanted as few library dependencies as possible.</li><li>I love C++, and I’m quite comfortable with it. I didn’t want to learn a whole new programming language <em>while </em>writing one<strong><em>.</em></strong></li></ol><p>If you decide to write a language (please do), you can write your language in any language of your choosing. There is no hard and fast rule, as long as the language you pick is Turing Complete, or at least allows for all the required constructs to implement your language design. Heck, you can even write a programming language even in Bash (please don’t).</p><h3><strong>The C++ version question</strong></h3><p>C++ has a vast number of compilers, and the standards committee releases new versions of the C++ standard regularly. But there’s a catch. For the sake of stability (Ubuntu comes to mind), most operating systems don’t include implementations of the latest C++ standard. Therefore, if I used one of the newer C++ versions, it would drastically reduce the accessibility of my language on systems that still use older compilers. Considering that, I went with using C++11 standard, as it is available on most computers and provides a good set of features.</p><h3>Feature satisfaction</h3><p>After finalizing the language I was going to use, it was time to think about how I was going to implement the features I set when designing it.</p><p>This is where my previous understanding of C++ really helped. I was able to figure out quickly how the features I had thought of could be implemented:</p><ol><li><strong>Extensibility</strong>. Using C++, I could create dynamically loaded modules (DLLs) which would contain specific extensions for the language (plugins) and would be loaded on demand when the source code required it. This is accomplished by using shared objects in C++ for plugin functionality.</li><li><strong>Performance. </strong>Well, unless we are talking about C or Assembly, I don’t know any other language which is as fast as C++ (and available on so many platforms). Also, with the use of C++ plugins (DLLs), I could easily write high performance parts in C++ and use them directly in Ethereal.</li><li><strong>Turing Completeness. </strong>There is no going around it. I would have to implement all the required control structures — which I wanted to do anyway — to make the language general purpose.</li><li><strong>Ease of Use</strong>. Overall, I chose the syntax similar to that of C and Python. I have a dislike of Python’s indent-based blocks and the inability to write one-liners. I wanted that in my language, so I used brace delimited blocks like C, semicolons for statement termination, and completely discarded the need for indentation (<em>although I implore people to write properly indented code for readability</em>).</li><li><strong>Modularity</strong>. I made the language modular by enabling other source files to be imported into each other. You can simply create a feature in one source file and import it in your main source.</li><li><strong>Basic stuff</strong>. All the required constructs are implemented and I created a special subscript function [] for enabling the extension of the subscript operator by types other than Vectors and Maps, without having to hard-code it in the language.</li></ol><p>I also thought about an important question…</p><h3>To OOP or not to OOP</h3><p>I’ll be honest. I have a love/hate relationship with object-oriented programming. On one hand, it gives you a helpful way (classes) to put data and the related function in a single unit. Inheritance and polymorphism have their uses too — I’ve even used them quite a bit in the Ethereal language codebase.</p><p>On the other hand, OOP makes the code look much more complicated and can be overkill for smaller things. I don’t want to write a full-blown class-object program for a simple script!</p><p>To that end, I thought of the most awesome feature I like about OOP — the member functions of course! I decided to implement them, but discard all the other OOP features. (<a href="https://en.wikipedia.org/wiki/Alan_Kay">Alan Kay</a>, don’t @ me.)</p><p>Even the design of member functions was easy. I just <em>bound</em> simple functions to specific types and allowed calling those functions from the respective type variables. The functions could get a reference to the calling type variable with the self keyword. This also made them very efficient, since there was no complex computation going on behind the scenes.</p><p>That’s literally it. This enabled me to use the elegant syntax of member functions, while not having to implement OOP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/1*NEI6_YBoOta14cRb4juLCQ.png" /></figure><h3>Fun stuff: Some Ethereal code</h3><p>At this point, you may be wondering about how I use the language and eager to see a tiny bit of what it can do. Well, here is a piece of code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*WNccrZvKVJOonkgMlwndlw.png" /><figcaption><strong>perform_tests.et</strong> — Execute all tests and see if they pass or fail</figcaption></figure><p>This code (located in the <a href="https://github.com/Electrux/Ethereal">main repo directory</a>) is the test driver for executing all the tests in the <strong>tests/</strong> directory in the source code tree. It produces a count of passed and failed tests, along with how much time (in milliseconds) it took to execute each of those tests, and then exits using the number of failed tests as the exit code.</p><p>This is helpful because if the exit code is not 0 (i.e. failed count is not 0), continuous integration systems (like Travis and Cirrus) will mark the build as failed.</p><p>In essence, this code first finds the list of files in the tests directory with <strong>.et</strong> extension, initiates a counter of passed and failed tests, loops through the list of files, and executes them. It shows the duration it takes for the execution of each one. Finally, it shows the OS, number of passed/failed tests, and the total time it took for the execution.</p><p>Here is a screenshot of the execution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*WyAr2BjU1mg4a1tkiqijCw.png" /><figcaption>Executing <strong>perform_tests.et</strong></figcaption></figure><h3>Conclusion</h3><p>That’s it for this article. I honestly had a hard time deciding what to put in and what not to include because of the vast amount of possible content. I hope it’s informative and not too overwhelming.</p><p>In the upcoming articles, I’ll be writing about the individual components and features in more detail. I still have a lot to discuss about the language.</p><p>I hope you liked the article. I love to get feedback regarding the articles and the language itself, so be sure to send them my way. And if you’re planning to write a language of your own (or have already done it), drop a comment to let me know!</p><p>Thanks a bunch for reading, and until next time! ❤️</p><p>Ethereal GitHub: <a href="https://github.com/Electrux/Ethereal">https://github.com/Electrux/Ethereal</a></p><p>Previous Article: <a href="https://medium.com/young-coder/so-i-created-a-programming-language-4d9c11038d22">So, I created a Programming Language</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad0b84bb04e3" width="1" height="1" alt=""><hr><p><a href="https://yc.prosetech.com/architecting-the-ethereal-language-ad0b84bb04e3">Architecting the Ethereal Language</a> was originally published in <a href="https://yc.prosetech.com/">Young Coder</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[So, I Created a Programming Language]]></title>
            <link>https://yc.prosetech.com/so-i-created-a-programming-language-4d9c11038d22?source=rss-852839018f8a------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d9c11038d22</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[programming-languages]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[virtual-machine]]></category>
            <dc:creator><![CDATA[Chirag Khandelwal]]></dc:creator>
            <pubDate>Thu, 29 Aug 2019 04:45:20 GMT</pubDate>
            <atom:updated>2019-10-03T20:44:56.909Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://medium.com/young-coder/ethereal/home">Creating a Programming Language</a></h4><h4>Deep down, you know you want to make one too</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Ey4XLqtGCjiSDf8zW0rZg.jpeg" /><figcaption>Image by <a href="https://pixabay.com/users/Free-Photos-242387/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=768432">Free-Photos</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=768432">Pixabay</a></figcaption></figure><p>It still feels weird (and awesome) to say that I actually created a programming language. Here’s a bit of my journey, and an overview of the language design for <strong>Ethereal</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*LM7gJkWXhBvXTbOIAP5Kww.png" /><figcaption>Ethereal — Simple Factorial Program</figcaption></figure><h3>Some background</h3><p>I’ll be the first to admit that I’m probably not particularly good at systems programming. I’m not an expert at language design, and I didn’t have a clue about how to create a language. What I did have was a really strong interest in making one. Compilers, Interpreters, and Languages have fascinated me for a long time and I always wanted to know how it would feel to be writing code and realizing that the code is written in my own language. (Spoiler alert: It feels <strong><em>GOOOD</em></strong>!)</p><blockquote>The compilers and interpreters like GCC, LLVM, CPython, etc. are truly exquisite pieces of code which — even to me — are totally mind boggling. But building your own language is not impossible.</blockquote><p>I have been writing small and esoteric languages (<a href="https://github.com/Electrux/Alacrity-Lang">Alacrity Lang</a>, <a href="https://github.com/Electrux/CCPP-Code/tree/master/ElectruxShorthandInterpretedLanguage">ESIL</a>) for about a year and a half now, but they have all been very limited in their capabilities, possibilities, and architecture. But this time, I wanted a (mostly) general purpose interpreted language. And so <a href="https://github.com/Electrux/Ethereal">Ethereal</a> was born.</p><p>I started working on this language from sometime in May 2019 and, over time, built a Lexer, Parser, Bytecode Generator, and a stack-based Virtual Machine for running the generated bytecode. I will be working on these components as needed as well as building various modules for it. It’s been nearly 4 months since I started implementing this language and, to be honest, it’s been an amazing journey.</p><h3>Language overview</h3><p>Ethereal is an interpreted language which translates the given source code to bytecode and runs it on a custom stack-based Virtual Machine. Its syntax is inspired from C and Python and is built on the fundamentals of being simplistic in nature like C, while having aesthetically pleasing syntax like that of Python. Personally, I’ve always preferred using braces to define blocks of code instead of indentation, and that’s what I’ve used here. Internally, Ethereal doesn’t care about lines or indentation for reading the source code (similar to C).</p><p>As of writing this post, I would honestly not even consider Ethereal to be in beta stage right now (it’s probably somewhere in early alpha stage). Coming to beta stage will probably take a while. But for now, it supports the following features:</p><ol><li>Variables, Functions, Conditionals, Loops, Enums, Structures — the usual</li><li>Member functions (in a way) for structures — functions that work for specific structures/types and are invoked using the dot operator (like in OOP languages)</li><li>Functions and types in C++ (or C) using modules, or in Ethereal itself using structs and imports</li><li>Function overloading — based on argument count and types for internal (module) functions; and based on argument count for functions created in Ethereal</li><li>Dynamic typing</li><li>Standard Library consisting of basic console and file I/O and crucial data structures — Vectors, Maps, Sets, Optionals (similar to pointers)</li></ol><p>It supports Linux, macOS, BSD, and Android operating systems out of the box (see Git repo for more info) and there are plans for possible Windows support.</p><h3>Why interpreted?</h3><p>The question that I get asked most often is why I chose to build an interpreted language instead of a compiled one. I originally thought about this quite a bit. I realized that I could afford to sacrifice some of the performance that compiled languages give to make an interpreted language. If I created a compiled language, I would have most likely used LLVM APIs to create the backend of the compiler (the optimization and code generation). But for this project, I wanted to create the entirety of the implementation myself (and I didn’t want to indulge in creating assembly code just yet).</p><p>Finally, building a Bytecode Generator and Virtual Machine seemed like more fun!</p><h3>Language for the interpreter</h3><p>I enjoyed using C when I started programming, and when I switched to C++, there was no going back. So, for me, language choice was more or less nonexistent. I love C++ and chose it as the language for building the reference interpreter in a heartbeat. Since it inherently provides high performance (provided one writes decent code), I wouldn’t need to worry too much about my code running slow.</p><p>Also, it is far better to write the interpreter for a language in a compiled language. Interpreted languages are slower than compiled ones. Creating an interpreted language in another interpreted language (like Python, Ruby, etc) would be too staggeringly slow.</p><h3>Components of the language interpreter</h3><p>Let’s take a small sample program:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/1*CfKfDBEHuqf28l8UwRuJQw.png" /></figure><p>The language interpreter consists of 4 fundamental components:</p><p>1. Lexer — Converts the provided source code into individual, separate tokens recognizable by the language</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/409/1*Lr3F0cwfWqlBVGUQ9KSLDQ.png" /><figcaption>Lexical Tokens — Sample Program</figcaption></figure><p>2. Parser — Converts the sequence of tokens into a parse tree which provides a meaningful format, to the written code, in terms of Ethereal</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/465/1*uxyoGGXBWQZ9HtywguIgAg.png" /><figcaption>Parse Tree — Sample Program</figcaption></figure><p>3. Bytecode Generator — Converts the parse tree to a sequence of custom bytecode instructions</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*iPP-BVWM-mBZXdSGJ-adHg.png" /><figcaption>Bytecode — Sample Program</figcaption></figure><p>4. Virtual Machine — Executes the generated bytecode sequence to produce the final result/output (here, <strong>15</strong>)</p><p>Personally, in ascending order, the difficulty in creating these components went as follows:</p><ol><li>Lexer — Rather easy</li><li>Bytecode Generator — Mostly okay, had a hard time generating bytecode for expressions</li><li>Parser — A bit more difficult, especially in parsing expressions, but also in general</li><li>Virtual Machine — Took the longest, mainly due to all the internal components like memory management, variable (and scope) management, and so on.</li></ol><h3>Dynamic library modules</h3><p>In this language, I also added support for dynamic modules — functions and variable types written in C++ itself. For instance, the functions like println and operators (+, -, and so on) are implemented in C++ as library functions which are then loaded by the Virtual Machine (Dynamic Library Loading). This is how (almost) all of the standard library for the language is built.</p><p>This feature allows one to create performance critical components in C++, therefore, not bottlenecking the program. Another benefit of this is the fact that one can easily create interfaces to third party C/C++ libraries (say Audio, Network, Graphics) for Ethereal, thereby, greatly extending the language.</p><h3>Reference counting</h3><p>Ethereal does not use a garbage collector. It simply doesn’t need one because all the variables are reference counted — the counter is increased when the variable is used, and is decreased when it is no longer in use. Once the counter for a variable hits zero, the variable object is deallocated.</p><p>This is a very simple architecture both in design and implementation, which provides quite consistent performance. I did implement a basic garbage collector originally, but it blocked the execution (for some milliseconds depending on program) whenever it had to free the unused objects. To improve performance, I trashed it completely and switched to reference counting.</p><h3>Overloadable operators</h3><p>By design of Ethereal, most of the operators (assignment = being the notable exception) can be overloaded for any particular type (only in dynamic library modules for now). If, for example, one is creating a module for vectors, they can overload the += operator for appending data to vector, + operator for joining two vectors, == operator for comparing two vectors, and so on.</p><p>Using this feature, the entire numeric arithmetic and boolean arithmetic core library is built. When we write something like a + b, it actually calls a function by the name of + for a &amp; b (see the Bytecode Generator output above — ID 3 to 6).</p><p>Aside from these features, there are a multitude more which can be explored by diving deeper into the language, but those are for another time.</p><h3>Conclusion</h3><p>Well, this was a bird’s eye view of my journey, as well as of the Ethereal language. I have a lot more to tell, but I’ll write a new post for that in the near future. I’ll also be creating a series going deeper in the technicalities of the language and how it all fell into place.</p><p>I hope it was an interesting read for you all. Language design is considered a very niche, nebulous, and complex topic, and — don’t get me wrong — it definitely is. The compilers and interpreters like GCC, LLVM, CPython, etc. are truly marvelous, genius, and exquisite pieces of code which, even to me, are totally mind boggling. But it is not impossible to do, and I love understanding all of it.</p><p>So if you are interested in language design and development, definitely give it a try. It is an incredible learning experience and absolute joy to work on. And don’t even get me started on the blissful feeling of writing code in your own language ❤️.</p><p>Below are the links to Ethereal and the other languages I worked on. If you’re interested, go ahead and get your hands dirty!</p><p>Thank you for reading!</p><p><em>Links</em>:</p><p><strong>Ethereal</strong>: <a href="https://github.com/Electrux/Ethereal">https://github.com/Electrux/Ethereal</a></p><p>Alacrity Lang: <a href="https://github.com/Electrux/Alacrity-Lang">https://github.com/Electrux/Alacrity-Lang</a></p><p>ESIL: <a href="https://github.com/Electrux/CCPP-Code/tree/master/ElectruxShorthandInterpretedLanguage">https://github.com/Electrux/CCPP-Code/tree/master/ElectruxShorthandInterpretedLanguage</a></p><p><strong>Next Article</strong>: <a href="https://medium.com/young-coder/architecting-the-ethereal-language-ad0b84bb04e3?source=friends_link&amp;sk=5336f28eead14f0e638e35632b814b91">Architecting the Ethereal Language</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d9c11038d22" width="1" height="1" alt=""><hr><p><a href="https://yc.prosetech.com/so-i-created-a-programming-language-4d9c11038d22">So, I Created a Programming Language</a> was originally published in <a href="https://yc.prosetech.com/">Young Coder</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>