<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by amit bezalel on Medium]]></title>
        <description><![CDATA[Stories by amit bezalel on Medium]]></description>
        <link>https://medium.com/@amit.bezalel?source=rss-f2d02b8804e0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*4Ni315mV_UGj2kHF.</url>
            <title>Stories by amit bezalel on Medium</title>
            <link>https://medium.com/@amit.bezalel?source=rss-f2d02b8804e0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:20:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amit.bezalel/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Everyone is Building an RCA Agent, We Built Ours Twice.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://pub.towardsai.net/everyone-is-building-an-rca-agent-we-built-ours-twice-6735f13be9fc?source=rss-f2d02b8804e0------2"><img src="https://cdn-images-1.medium.com/max/1400/0*DsHUkO7X2FcY4FGo.png" width="1400"></a></p><p class="medium-feed-snippet">A read-only agent that takes a customer complaint, digs through logs, data and code, and comes back with the root cause. Here are the two&#x2026;</p><p class="medium-feed-link"><a href="https://pub.towardsai.net/everyone-is-building-an-rca-agent-we-built-ours-twice-6735f13be9fc?source=rss-f2d02b8804e0------2">Continue reading on Towards AI »</a></p></div>]]></description>
            <link>https://pub.towardsai.net/everyone-is-building-an-rca-agent-we-built-ours-twice-6735f13be9fc?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6735f13be9fc</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Wed, 21 Oct 2026 13:01:06 GMT</pubDate>
            <atom:updated>2026-10-21T13:01:06.001Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Running any Container on CloudRun]]></title>
            <link>https://itnext.io/running-any-container-on-cloudrun-cca137cddbca?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/cca137cddbca</guid>
            <category><![CDATA[google-cloud-run]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Fri, 31 May 2019 23:28:43 GMT</pubDate>
            <atom:updated>2019-06-02T04:53:22.223Z</atom:updated>
            <content:encoded><![CDATA[<h3>DockerHub to Google Cloud Run</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZVOBM3MhqOncXKKwBRFmQg.png" /><figcaption>Blasting through to the cloud</figcaption></figure><p>A minute after hearing of it, I just had to try Google CloudRun out, turns out it has a small quirk: it requires your server to listen to a specific port, defined in the ${PORT} environmental variable. “Hay! what about running containers I just grabbed off of Docker-Hub, with no reason for modification !?”, Don’t judge me, as a citizen of this synchronized world, I have the god given right to be extremely lazy! and after all, the reason I liked the idea of CloudRun so much is because it enables my lazy streak…</p><p><strong>Let’s get down to business:</strong> I have some docker image that I want to run, it knows how to listen to a port which might be configurable in some config file or command-line argument, but it definitely knows nothing of the ${PORT} parameter that google injects into the container env.</p><p>A small look into the original dockerfile indicates it started from some Debian flavor (debian-slim, ubuntu, etc.) so I have apt get, you may need to use yam or apk depending on the distro.</p><p><strong>My plan:</strong></p><ol><li>Start from my target container</li><li>Install nginx</li><li>Change the nginx config to listen to the {PORT}</li><li>Config nginx to route traffic to the default port of the original server</li><li>Launch it on CloudRun and have a scalable cloud service up in seconds</li></ol><p><strong>You may have noticed a small problem</strong>: nginx doesn’t know how to listen to dynamic port numbers provided by env variables, no worries, we will used a small <strong>sed command </strong>in our container run script to replace the port in the nginx config before running it, along with the original server process.</p><p>Below is our nginx configuration. A basic server with a listener on port $NGINX_PORT (to be replaced by sed script on container run), and a proxy_pass command to send everything to the original server port (8042 here):</p><pre>worker_processes 1;<br>events { worker_connections 1024; }<br>http {<br>  server {<br>    listen $NGINX_PORT;<br>    location / {<br>      proxy_pass <a href="http://127.0.0.1:8042/">http://127.0.0.1:8042</a>;<br>      proxy_redirect off;<br>    }<br>  }<br>}</pre><p>The dockerfile just follows the plan, starting from the original image, installing nginx and configuring it, then running a script that will replace the nginx listen port with ${PORT} and run all required processes.</p><p>pay attention to the line with <strong>“Entrypoint []”</strong> as it removes the original entry point, eliminating errors or problems with the origial processes being launched in the container instead of our script.</p><pre>FROM jodogne/orthanc-plugins</pre><pre>RUN apt-get update &amp;&amp; apt-get install -y \<br>    nginx</pre><pre>RUN rm -rf /etc/nginx/nginx.conf<br>ADD nginx.conf /etc/nginx/nginx.conf<br>ADD run_nginx.sh ./run_nginx.sh</pre><pre>ENTRYPOINT []</pre><pre>CMD [&quot;bash&quot;, &quot;./run_nginx.sh&quot;]</pre><p>The only thing I now need from my original container is to take it’s original CMD (or Endpoint) and put it as the last process launched by my run_nginx shell file.</p><p><strong>Below is my nginx_run.sh file,</strong> it uses “sed” to replace the port, launches nginx as a background process, and then runs the original server as the main process for the container. If you have any problems first check if you have any[CR][LF] in your nginx_run.sh file and change them to [LF] so linux/bash doesn’t get mad…</p><pre>sed -i &quot;s/\$NGINX_PORT/${PORT}/g&quot; /etc/nginx/nginx.conf<br>exec &quot;/usr/sbin/nginx&quot; -c &quot;/etc/nginx/nginx.conf&quot; -g &quot;daemon off;&quot; &amp;<br>exec &quot;/usr/local/sbin/Orthanc&quot;</pre><p>Now all i need to do is build the docker, tag it and push it to a google cloud registry (CloudRun doesn’t work with other registries), and set up the service In the CloudRun console by selecting the docker image and enabling a public URL. optionally you can also provide any env variables you want to inject into the container, and change memory limits and timeouts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*oWVmQqgcES0kadNn4fIp8Q.png" /></figure><p><strong>Now press ‘deploy’ and after a minute, check that the service is up and running by calling its Url. </strong>This new service is now a fully scalable service since google will run additional containers whenever it gets hit with high traffic, this is done by checking the max concurrent requests per container in the settings (currently, 80 is the highest setting allowed). Billing on a container is only done when at least one request is active, so a running container which is not accessed costs absolutely nothing.</p><p>Hope this helped you use this very nice Goolge cloud service, it is about time scaling containers became easier to use than ECS/GKE/EKS and once you get over CoudRun’s single rough spot it does run your container like pure magic.<strong> I definitely have to congratulate the Google cloud team </strong>for getting one over AWS, which is still my go-to cloud provider for most purposes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cca137cddbca" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/running-any-container-on-cloudrun-cca137cddbca">Running any Container on CloudRun</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Teleporter: Design your own network]]></title>
            <link>https://itnext.io/teleporter-design-your-own-network-af20d12454be?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/af20d12454be</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[socks5]]></category>
            <category><![CDATA[tunneling]]></category>
            <category><![CDATA[vpn]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Wed, 27 Mar 2019 07:58:26 GMT</pubDate>
            <atom:updated>2019-03-27T07:58:26.926Z</atom:updated>
            <content:encoded><![CDATA[<p>Did you ever wish you could have access to your home network while at work? It just got simple…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NEbK_ar_RZmGRX8K7BJTlw.png" /></figure><p>Recently I have been thinking about a project we did a year back, where we customized NGrok code to create a tunneling solution that will connect an AWS based browser to a client’s on premise web-server for a functional testing product (Microfocus SRF). I always thought that Protocol tunneling is an exciting concept, a cleaver way of creeping under the radar getting away with sending requests in the opposite direction, drilling holes through smothering protection and setting up impossible connections.</p><p>I thought of creating something similar that will help defend home automation setups, and meld together work &amp; home environments without using port mapping, setting up VPNs, or heavily affecting network performance. Teleporter is what I came up with, written from the ground up over muxado, <strong>and it is easy as pie to setup</strong>.</p><h4>Usage</h4><p>So before explaining any further, here is how to use it:</p><p><strong>For a demo on your local machine:</strong></p><ol><li>Download the latest build of <a href="https://github.com/amitbet/teleporter/releases/tag/1.0">Teleporter</a></li><li>Use “run.bat” in any of the examples to run several nodes on localhost</li><li>Configure your browser to use the socks5 proxy now running on localhost:10101 to see it in action.</li></ol><p><strong>To set up a real network:</strong></p><ol><li>Spin up some free tier cloud machine (GCP will give you $300 for a year, with no obligations, just by feeding in credit card info, AWS free tier machines are also an option)</li><li>Place a node on that machine &amp; open the relay port to the public</li><li>Deploy teleport nodes on your favorite machines &amp; configure (see below) to construct your own custom slice of internet!</li></ol><h4>The Configuration file structure:</h4><p>The config is designed to be mostly self-explanatory, it holds all the information for creating listeners, establishing connections and restricting network access. The default rule is to deny access, currently requests will be routed to the first node that allows the requested address.</p><p>The Teleport configuration file is split into 3 sections:</p><ul><li><strong>Servers</strong>: create listeners of 2 different types: socks5 proxy &amp; relay</li><li><strong>Tethers</strong>: define connections to relay ports on other nodes</li><li><strong>netConf</strong>: includes the node’s ID and mapping rules that allow access to resources reachable from this node, this part of the configuration is sent out to other nodes when connecting to them, this enables smart routing. the mapping rules are executed sequentially, they state which node will be chosen next for the route, choosing “local” means the request has reched it’s destination and should run on the current machine.</li></ul><pre>{<br>    &quot;servers&quot;: [<br>        {<br>            &quot;port&quot;: 10101,<br>            &quot;type&quot;: &quot;socks5&quot;,<br>            &quot;acceptLocalOnly&quot;: true,<br>            &quot;useAuthentication&quot;: true,<br>            &quot;authClients&quot;: {<br>                &quot;socks5User&quot;: &quot;&lt;SECRET STRING&gt;&quot;<br>            }<br>        }<br>    ],<br>    &quot;tethers&quot;: [<br>        {<br>            &quot;port&quot;: 10201,<br>            &quot;host&quot;: &quot;127.0.0.1&quot;,<br>            &quot;connectionType&quot;: &quot;tls&quot;,<br>            &quot;connectionName&quot;: &quot;Connection to node2&quot;,<br>            &quot;proxy&quot;: {<br>                &quot;address&quot;: &quot;&lt;proxy address&gt;&quot;<br>                &quot;user&quot;:&quot;&lt;optional proxy user&gt;&quot;<br>                &quot;pass&quot;:&quot;&lt;optional proxy password&gt;&quot;<br>            },<br>            &quot;password&quot;: &quot;&lt;SECRET STRING&gt;&quot;<br>        }<br>    ],<br>    &quot;netConf&quot;: {<br>        &quot;clientId&quot;: &quot;node1&quot;,<br>        &quot;networkMapping&quot;: {<br>            <strong>// send all *google* domain requests through node2</strong><br>            &quot;*google*&quot;: &quot;node2&quot;,<br>           <strong> //if not captured with prev rules - execute locally</strong><br>            &quot;*&quot;: &quot;local&quot;<br>        }<br>    },<br>    &quot;proxy&quot;: null,<br>    &quot;numConnsPerTether&quot;: 10<br>}</pre><h4>Potential Uses:</h4><ul><li>Stay connected to home equipment securely &amp; without port mapping</li><li>Seamlessly RDP/VNC into multiple networks for remote support</li><li>Create secure agent connections from customer sites to cloud services</li><li>Bridge network gaps without help from your IT department</li><li>Stay connected to work without using a VPN</li><li>Use as a custom VPN to spoof your origin, protect your privacy and gain access to location based services</li><li>Expose on-premise webservers to potential customers or cloud testing farms through a socks5 proxy and some relays</li></ul><h4>Security features:</h4><ul><li>Inter-node connections (tethers) are TLS encrypted</li><li>Socks5 connections can be password protected (although not encrypted)</li><li>Socks5 connections can be restricted to accept only from localhost</li><li>Relay Server Authentication (Client has a password in the server config)</li></ul><h4>Isn’t Teleport just another VPN?</h4><p>I have been pondering this same question for some time now: VPN, Tunneling and Socks5 have been around for years, has this idea been done before? well, it turns out that there are a ton of similar stuff out there: OpenVPN, BadVPN &amp; Brook just to name a few, but each is different in some essential aspect. But VPNs are essentially point to point tunnels bringing you into a network via an encrypted channel, as for Teleport:</p><ul><li><strong>Teleporter uses multiplexing over multiple TCP channels per tunnel</strong>, Most VPNs operate on a low network layer, others (like OpenVPN) rely on UDP as their main transport to avoid the <a href="https://en.wikipedia.org/wiki/Head-of-line_blocking">Head of Line problems</a>, there are implementations that support TCP tunneling, but use a single TCP connections and regard this a bad solution to be used in specific cases. TCP support enables tunneling through most corporate environment restrictions that are based on allowing only TCP on port 80/443 or using an HTTP Proxy, and using multiple connections to do so avoids blocking browser connections behind unacknoleged or lost TCP packets.</li><li><strong>Teleporter only relays requests for specified addresses, </strong>it creates an extremely split tunnel, giving the user full control over routing, this means that most of the requests can be handled locally, without lags or network costs. These specific address are your remote places (e.g. home), or addresses blocked by your work network (e.g: box.com, thepiratebay, etc.) there is almost no performance penalty for traffic that is executed locally and a small performance payment for sites that were never in your reach, allowing for home access without messy port mappings that expose your home machines to attackers on the public internet. Socks5 doesn’t use multiplexing, and is also unencrypted, which means no handshake delays or encryption overhead for local traffic.</li><li><strong>Teleporter easily creates multi hop networks </strong>which are not typical of VPN setups</li><li><strong>Teleporter is very easy to set up: </strong>just by changing several lines in a JSON file, running a single binary on several machines you can create a distributed network for yourself, set your browser or OS proxy to run via the local socks5 proxy, and you are done.</li><li><strong>Teleporter is built in Go</strong> so it easily compiles to all major OS flavors.</li><li>In Teleporter (future feature), rewrite rules can be used to have a “CNAME” like: home.com for your home machine address.</li></ul><h4>How does it work?</h4><p>Each Teleport node has the ability to create relay servers, socks5 servers, and to connect a connection-bundle (A.K.A. Tether) to other relay nodes, a Tether holds multiple TCP connections, each capable of TCP multiplexing. Each such tether is bi-directional, allowing requests to be initiated by either side.</p><h4>The Full Story</h4><p>Our original solution was based on NGrok 1.0, with an added http-proxy server on the browser machine, which meant that for each browser connection, we were creating a connection from the proxy to the tunnel-server, from the tunnel-client to the tunnel-server and from the tunnel-client to the web-server. after we built it and started working on connection caching and reuse, I thought “this can be better solved by multiplexing”, but when digging into the subject, I found out that since TCP requires an ACK for each packet, it tends to get stuck (the <a href="https://en.wikipedia.org/wiki/Head-of-line_blocking">head of line problem</a>), and that UDP should be used for such solutions, but UDP will not pass out of most corporate setups, and never pass over a WAF into an AWS farm… I decided to drop the subject, but some corner of my mind kept nagging at me that there should be a better way.. So I created <strong>Teleporter</strong> as side project to better explore the possibilities.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af20d12454be" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/teleporter-design-your-own-network-af20d12454be">Teleporter: Design your own network</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My first course with Packt publishing]]></title>
            <link>https://medium.com/@amit.bezalel/my-first-course-with-packt-publishing-c5da900a1c54?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5da900a1c54</guid>
            <category><![CDATA[course]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[writing]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Tue, 26 Mar 2019 20:00:49 GMT</pubDate>
            <atom:updated>2019-03-26T20:00:49.053Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AMPPqQoJMAsOwc9rA1nodw.png" /></figure><p>My first course: “<a href="https://www.packtpub.com/application-development/hands-microservices-typescript-3-video">Hands-on Microservices with Typescript</a>” is out, it took 3 months to make here is what I learnt in the process.</p><h4>Meeting up</h4><p>One day, while skimming my Gmail messages, I came across a mail from Packt. Generally dismissing it at first glance as some advert, I thought “why didn’t google filter this out?”, the message body provided an explanation: Someone at Packt read my medium posts, and asked me if I was interested in creating a course with them. Naturally, I was still suspicious since I, like most computer professionals, am hard wired to think of random mail approaches as scams, but the offer sounded genuinely directed towards me, mentioning posts that I have written, so I replied.</p><p>Turns out it was the real deal, in the following Skype conversation they suggested several course ideas, and I agreed to make a Microservices course, which is a topic I especially like, since I pioneered building in this architecture in HPE Software, and had a ton of fun learning all the corners while doing so.</p><h4>Signing up</h4><p>Packt are a large scale publisher, which means they have everything in order, there is crew for content acquisition, and you work with someone on the other end that you can approach with any questions or requests. There are instruction videos for getting started, recommended hardware, an author portal, and a standard contract to sign in order to start creating the course.</p><h4>The process</h4><p>Building an online course contains several different tasks, some of which, like presentation building, I already know well:</p><ul><li>Building the presentation while using only free images</li><li>Dividing the content into 5 sections, around 4 videos each, which should be ~5–10 minutes long.</li><li>Narrating the course slides while screen capturing</li><li>Capturing &amp; narrating hands-on instruction videos</li><li>Creating a multiple-choice quiz with 25+ questions and explanations about the answers</li><li>Creating a code repo with all the examples</li></ul><p>I was surprised to find I had a lot to learn: getting a good narration setup took some work, I needed to tweak screen capture parameters, understand how to speak to an unseen audience &amp; get the equipment setup. The recommended microphone (blue snowball) was an ambient mic, and while trying out several different mics and setups I found out that there is a trade off: ambient mics are very good at capturing sound, but they also amplify background noise, any hums buzzes or outside honks get recorded as well as echoes from the walls. headphone mics totally ignore such noise, to the point where I had the upstairs neighbor knocking nails into the wall, then playing it back and being surprised that what got recorded aside from my voice were some barely audible thuds. headphone mics are more sensitive to mouth pops, since they have to be closer to your face, and are generally smaller which means they have less overall recording quality, I decided to go with headphones, with a nice foam filter to avoid popping sounds.</p><p>For screen capturing they recommended <a href="https://obsproject.com/">OBS</a> (Open Broadcast Studio) which is free, open source and very good. After tweaking the settings to use the CPU’s hardware acceleration for video encoding, I got pro level results. I saved the video in seamless quality since the video will be edited and compressed before posing online.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aqOHfKsrODd6Ai8yudkqiw.png" /><figcaption>OBS screen capture</figcaption></figure><p>Building up the course material was fun, I got to organize all the stuff I researched in the last 4 years, I built the demo application from the ground up with docker compose to hold it all together, adding comparisons and recommendations to help guide people over the many decisions that need to be made in the process of designing such a project. Since this course has a lot of theory pages as well as code, and since text filled slides always get me snoring while in the student’s chair, I tried to add as many images and illustrations as I could. I also did a lot of reading to deepen and verify everything generally making sure I don’t mislead any of my students. There is truth in the saying: “you learn best while teaching” teaching students is a surefire way of tightening and proofing your knowledge in any field.</p><p>While narrating the slide videos I took their advice and scripted it. Reading stuff aloud without thinking about the meaning somehow felt like cheating at first, but it produced better results, and I wrote it after all so what the heck… Talking over a slide is easily editable since the screen doesn’t change, this means that if you mispronounce a sentence, you can just say it again and cut out the previous attempt without a noticeable seam. I also heard from other vid-course authors that it helps to clap loudly each time you mess up, this leaves an “audio bookmark”, a nice spike in the audio graph which shows them exactly where to cut.</p><p>For the Code Demo videos, there was no escape, and long stretches of video had to be captured in a one long block, this meant several takes until I was happy with it and sometimes getting tired of the repetitions and just keeping something I convinced myself was okay even though I sounded funny in some sentence. Another thing I did with the code shots was to prepare all code in advance. I decided I usually don’t enjoy online coding videos, and tend to skip to the end, pause and read the code itself, there is no reason to watch someone typing a book you want to read, you just pick up the finished title… same with watching someone typing code.</p><p>The code examples were a place I was totally confident in my abilities, but as it turned out, since the code was incrementally built, every time I found something I wanted to fix in the basics (adding complexity to base layers in order to demonstrate something, finding a bug, cleaning up for readability, etc.) made me go back and integrate that fix into each previous example, which is tiresome code-merge work, especially when you have 20 such examples. I haven’t found a way around this other than “<strong>write no bugs</strong>” and “<strong>think of everything in advance</strong>”, maybe writing the samples backwards would help, by first creating the full application and then cutting it back to get simpler samples, but I think this method would leave some added complexity in the simpler stages. (tell me if think of any good way of solving this)</p><p><strong>Here is an image of it in the packt site (kinda looks like I put a paid ad in my medium post doesn&#39;t it? well technically…)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*XRSP_fX2Ig1ZgOze5SbWuA.png" /><figcaption><a href="https://www.packtpub.com/application-development/hands-microservices-typescript-3-video">My course</a> on the packt site</figcaption></figure><p>The end result is a 3 hour course which I feel proud of making, it took much more effort than I initially thought I would invest, but it was well worth it. I hope people will view it and benefit from it, and I would be happy to receive any feedback.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5da900a1c54" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[QuickFix: Vs-code Golang Renaming]]></title>
            <link>https://itnext.io/quickfix-vs-code-golang-renaming-b191350dd02b?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/b191350dd02b</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[vscode]]></category>
            <category><![CDATA[gorename]]></category>
            <category><![CDATA[rename]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Mon, 07 Jan 2019 06:23:16 GMT</pubDate>
            <atom:updated>2019-01-09T08:12:49.863Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1dvfGm8Df9OO3n2C1pCLQ.png" /></figure><p><strong>Golang Rename in vs-code is unusable, Here’s a fix for it!</strong></p><p>Rename is an essential tool in any coding language and vs-code is a very popular IDE for Go, so it is surprising that this issue was never solved. When looking it up online, I found out that the vs-code plugin for golang just runs golang’s own rename tool. Golang is fairly fast, and most golang tools are properly constructed, So why is it so slow?</p><p>The problem is that this tool often thinks it needs to search through the *<strong>Entire GOPATH* </strong>for any dependencies that might need renaming, <strong>which is never what anyone pressing F2 is expecting</strong>. Searching the git issues, I found that there is a fix out there, @github.com/parteek already commented out the global search code, but when compiling his gorename I still got the same issue, and there was no compiled binary. Turns out it somehow still uses the golang.org/x/tools/refactor/rename package when I build it…</p><p>So I thought I’ll make it easier, I reorganized the code into one package, and tested that it works. comparison with go 1.11 code shows that the “tools/refactor/rename” code hasn’t changed since.</p><h3>installing:</h3><pre>go install github.com/amitbet/gorename</pre><p>You should now have a new gorename.exe in your $GOPATH/bin directory.</p><p>Now renaming in vs-code works within two seconds (due to code checking and vetting done before running gorename)</p><p><strong>Original hack: </strong><a href="https://github.com/prateek/tools/commit/72b0e988ce6957db328582decda8829154c66545">https://github.com/prateek/tools/commit/72b0e988ce6957db328582decda8829154c66545</a></p><h3>Any Issues?</h3><p>If you is still get a 100% CPU gorename.exe process, you are using it from the wrong directory, check the process&#39; command line in task manager, or run</p><p>Windows:</p><pre>where gorename.exe</pre><p>Or on a mac:</p><pre>which gorename.exe</pre><p>Then change your PATH variable so the right gorename.exe is invoked, or overwrite the other one with the new one in your $GOPATH/bin directory.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b191350dd02b" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/quickfix-vs-code-golang-renaming-b191350dd02b">QuickFix: Vs-code Golang Renaming</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Language awards: war of codes]]></title>
            <link>https://itnext.io/language-awards-war-of-codes-d751eb386184?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d751eb386184</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Tue, 02 Oct 2018 06:39:24 GMT</pubDate>
            <atom:updated>2018-10-02T06:39:24.875Z</atom:updated>
            <content:encoded><![CDATA[<p>Which language is best for you? TS, Javascript, Go, Java, Py, C# Let’s go deep and find out!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w5msDVay2Nm3_UfzlgqlWw.png" /></figure><p>Some time after the initial TS vs Go comparison I thought I should revisit the language comparison to do something bigger, I wanted to go deeper and broader, picking at some pain points, comparing syntax and functionality and ecosystems, as well as throwing more languages into the mix, to see how the newcomers stack against the older languages. Why not compare everything in a big ol’ table, and see who gains the most points? Ok. It will be messy, and you might not like my grading, but I promise I’ll try to be fair ;)</p><p><strong>TL;DR: </strong>Consider the following table, feel free to fight it out in the comments:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pnCAVZp10-knPHM-SgxnrQ.png" /><figcaption>My grades for the language comparison</figcaption></figure><p>Now that you’ve got your spoiler, let’s get on with the comparison. I won’t go into Java / C# / Py <strong>parallel</strong> code samples as I want to focus on TS &amp; Go in this area, but such samples are not hard to find.</p><h3>Resource consumption</h3><h4>Memory footprint</h4><p>Memory utilization varies widely when using different languages, this might sound surprising at first, since you may think that the code should use up approximately what the algorithm requires, but since the data can be presented in many different ways in memory language and base library implementations make a huge difference in memory usage. For example: objects can be represented as pointer networks (C#/Java), sequential structs (Go) or hash maps (JS).</p><p>The chart below shows Memory consumption during different code tasks, a cutoff of 850MB was introduced to retain graph detail against over scaling, C# code benchmarks are of .Net core performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*9A-eFhYC4cKQ1UnuvnK95A.png" /><figcaption>Memory benchmarks (data: the benchmark game)</figcaption></figure><p>When speaking about <strong>Garbage collected</strong> languages, <strong>Golang is your best bet</strong> for top memory utilization, and this can also be seen <a href="https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/go-node.html">in benchmarks</a>:</p><ul><li><strong>Memory sequential structs</strong>: in contrast with most languages (including JS) Go has no classes, it only has structs. This means that Golang objects are encouraged to be more like blocks of binary data than a tree of pointers, I say “encouraged” since you could use pointers for everything in Go if you wish to do so.</li><li><strong>Pure code compilation</strong> vs linking (dynamic &amp; static) means that you don’t need to load the whole *.dll/*.sa/*.js file into memory for a few function calls, think of using lodash, how many code lines are loaded vs used.</li><li><strong>Strong types</strong>: precise control over memory usage e.g. int16 only uses 16 bits.</li><li><strong>No Inheritance</strong>: Golang has no need for virtual tables, which means you reduce the overhead for almost every object created, since in Java/C# inheritance is the norm.</li><li><strong>Multi-Threading</strong>: With JS you will need to create new processes, to utilize additional CPU cores, which means that you load duplicate copies of base classes and library code.</li></ul><h4>CPU Utilization</h4><p>We programmers have a hobby of asking which language is better at squeezing out the juice, and I am no exception. With each new language I have learned, I always think: “Is this one much faster than the previous few?”, picking a clear winner in this category will always be a scam, but I can offer some thoughts on the matter.</p><p>We can turn to <a href="https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/go-node.html">benchmarks</a>, which show us that Golang will most likely outperform JS, which is not surprising since more cores can be used, but it is a different story when comparing it to <a href="https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/go.html">Java</a> &amp; C#.</p><p>The chart below shows the time to complete different benchmarks, I had to introduce a cutoff of 90 secs, to avoid python times washing out the more important results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*P6nLzF0tOjMuN5ElmkGctg.png" /><figcaption>CPU benchmarks (data: the benchmark game)</figcaption></figure><p>I know that benchmarks are “artificial”, statistics lie, X didn’t check gcc-go, and so on. My thoughts on the matter is that like most things in life, this is a trade off. Golang is geared towards servers, it has a fast, single gen GC which sacrifices object creation/removal speed for sub-millisecond “stop the world” pauses, this means that serving will be quick and without strange GC hickups, as shown in <a href="https://www.techempower.com/benchmarks/#section=data-r16&amp;hw=ph&amp;test=fortune">server benchmarks</a>, where go-fasthttp comes in at #4 out of 307 while nodeJS is #77 and while Java appears on #3, vert.x is not yet a popular way to write java servers, undertow is #19 and to find jetty you would need to go down to #78.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-P-xNKQ2btz0nrlhHgMruw.png" /><figcaption>Source: <a href="https://www.techempower.com/benchmarks">techempower.com/benchmarks</a></figcaption></figure><p>This GC strategy in Golang also means that when creating a <strong>huge</strong> amount of short lived instances, Golang will be slower, as shown in the <a href="https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/binarytrees.html">bin-tree benchmark</a> where a huge amount of nodes are created to construct large tree structures: Golang is 11 times slower than C, while C#.netCore (3.1 times), Java (3.4 times slower), NodeJS (9.2 times). Proof of this issue can be seen <a href="https://blog.kowalczyk.info/article/u5o7/speeding-up-go-with-custom-allocators.html">here</a>, by using custom allocation which avoids GC thus fixing the performance problem, I do not believe most normal usecases create similar situations, so Golang should be more than ok with handling your server workflows.</p><p>In most cases, code implementation specifics count much more towards real world performance than anything language specific, but if you want to get picky (and we do), Golang is a solid bet for performant code seeing that it won 1st place in 5 out of 10 benchmarks, 4 going to C# and 1 to Java.</p><h4><strong>Async &amp; parallel programming</strong></h4><p><strong>Java: old threadpooling paradigm, </strong>uses OS threads, allows self management and some pooling abilities, sync primitives like mutex and semaphore are provided as well as synchronized access to functions. Async IO was introduced in recent versions, but Java lacks some modern approaches which should make this messy field easy to implement and stir you away from thread leaks, dead locks and a slew of other parallel coding horrors.</p><p><strong>C#</strong>: <strong>Some good approaches for sync and parallel, a bit confusing at times<br></strong>C# has all the parallel methods they could give it, OS threads, threadPools, sync primitives and advanced sync objects as well as Tasks scheduling (TPL), but it isn’t very straight forward: tasks can be concatenated by using continueWith, which is allows for a more convenient approach to parallel execution of code but task preemption is not always handled by the framework so an http request can still occupy an OS thread until timed out or response is received, in order to ensure asyncIO, the newer http client’s async/await methods should be used which means the rest your code should also be async… the correct usage is not as clear or as simple as should be, as a result of the many different async &amp; parallel options within the framework. For Async programming we have the Async/Await functionality, which works well while you don’t mix it with fat clients which require synchronous UI threads. As you can understand C# tries very hard to give us everything… but it is lacks the smoothness of something that was built from scratch with the benefit of hindsight.</p><p><strong>TS: Simplicity, no parallel, always async code<br></strong>NodeJS takes the JS approach in which there is a single thread for processing JS commands, and offloading all IO work to async IO threads managed by the OS, to carry out any transactions with the Disk/Network. the main thread is managed by a task queue which means that at the end there is only one JS command being handled at any time in the NodeJS process. This means that your code is always async, removing the problem of settling sync code and sticky async code which tends to infect all adjacent code with it’s async way of life.</p><p>This “one thread” mode of operation was very surprising to me when I first heard of it, since servers are all about efficiently using all cores to maximize calls per second handling. But if you think of it a bit longer, some prominent DBs used multi-process architectures for a long time, so if the overhead per process is not large, it can make sense to have a good “single thread per process” management scheme, and NodeJS succeeds in providing just that.</p><p>The downside is that your code is completely written in an async way, both callbacks and promises have a learning curve, using async/await is much better but you still need to remember that your code is being paused, and plan accordingly.</p><p><strong>GO: Parallel Power, Sync code until requested<br></strong>Golang gives you much more control over parallel and async flows, and you may argue that this was already the case with previous languages like java/C#, but when you see that a million go-routines can be run simultaneously without impacting system performance, you realize that there is something new here (that auto-preemption I was talking about)…<br>Goroutines are <a href="https://codeburst.io/why-goroutines-are-not-lightweight-threads-7c460c1f155f">not threads</a>, they take up only 2k stacks and are held within GO’s managed memory pool, so both creating them and disposing of them are incredibly fast actions. go also provides you with Channels which are producer-consumer queues that are used to tie go-routines together and offer a simplified solution to most synchronization problems, steering you away from those pesky sync catastrophes.</p><p>After talking about power we should still think of simplicity; In Golang writing sync is simple, but when going async there is a learning curve, you need to think of “the Go way”: how to correctly create go-routines, pass channels, execute select statements and check that routines close properly when done. In short,despite Golang’s superior parallel facilities, You would still not want to leave async/parallel code design up to your weakest programmers.</p><p>Check out this cool example of Golang’s power:</p><pre><strong>100K-long goroutine chain: (~600 Milisecs on my machine)</strong></pre><pre>package main<br>import &quot;fmt&quot;</pre><pre>func f(left, right chan int) { left &lt;- 1 + &lt;-right }</pre><pre>func main() {<br>  leftmost := make(chan int);<br>  var left, right chan int = nil, leftmost;<br>  for i:= 0; i&lt; 100000; i++ {<br>    left, right = right, make(chan int);<br>    go f(left, right);<br>  }<br>  right &lt;- 0;  // bang!<br>  x := &lt;-leftmost;  // wait for completion<br>  fmt.Println(x);   // 1000000<br>}</pre><h3>Code writing experience</h3><h4>Serialization</h4><p>serialization support in TS just uses JS, with the additional ability to have interfaces defined which outline the serialized objects, allowing for easy code completion support. It is by far the easiest serialization experience I have had in any language. GO serialization support is nice too, but it requires either defining and filling structs along with some serialization notations to indicate the json field name, or using Goalng’s unknown type “interface{}”:</p><ul><li><strong>map[string]interface{}</strong> //A map of unknown type</li><li><strong>[]interface{}</strong> //An Array of unknown type</li></ul><pre><strong>Using Object Definitions:</strong></pre><pre><strong>TS:<br></strong>class ChildStruct {<br>    Text: string;<br>    SomeField: string;<br>}<br>class SomeStruct {<br>    isAFoo: boolean;<br>    someField: string;<br>    children Array&lt;ChildStruct&gt;;<br>}<br>var p0 = new SomeStruct(true, &quot;blah&quot;, [{&quot;aa&quot;,&quot;bb&quot;},{&quot;cc&quot;,&quot;dd&quot;}]);<br>var s = JSON.stringify(p0);</pre><pre><strong>GO:</strong><br>type ChildStruct struct {<br> Text     string         `json:&quot;text,omitempty&quot;`<br> SomeField string        `json:&quot;someField,omitempty&quot;`<br>}<br>type SomeStruct struct {<br>   IsAFoo   bool           `json:&quot;isAFoo&quot;`<br>   Text     string         `json:&quot;text,omitempty&quot;`<br>   Children []*ChildStruct `json:&quot;children,omitempty&quot;`<br>}</pre><pre>jstr, _ := json.Marshal(SomeStruct{<br>    IsAFoo: true,<br>    Text: &quot;blah&quot;,<br>    Children: []*ChildStruct{<br>        &amp;ChildStruct{<br>            Text:      &quot;aa&quot;,<br>            SomeField: &quot;bb&quot;,<br>        },<br>        &amp;ChildStruct{<br>            Text:      &quot;cc&quot;,<br>            SomeField: &quot;dd&quot;,<br>        },<br>    },<br>})</pre><pre><strong>Inline Serialization:</strong></pre><pre><strong>TS:<br></strong>let obj= {<br>    &quot;isAFoo&quot;: true,<br>    &quot;text&quot;:   &quot;blah&quot;,<br>    &quot;children&quot;: [<br>       {<br>            &quot;text&quot;: &quot;aa&quot;,<br>            &quot;someField&quot;: &quot;bb&quot;,<br>        },<br>        {<br>            &quot;text&quot;: &quot;cc&quot;,<br>            &quot;someField&quot;: &quot;dd&quot;,<br>        },<br>    ],<br>}<br>let str = JSON.serialize(ob);</pre><pre><strong>GO:</strong><br>obj := map[string]interface{}{<br>    &quot;isAFoo&quot;: true,<br>    &quot;text&quot;:   &quot;blah&quot;,<br>    &quot;children&quot;: []interface{}{<br>        map[string]interface{}{<br>            &quot;text&quot;: &quot;aa&quot;,<br>            &quot;someField&quot;: &quot;bb&quot;,<br>        },<br>        map[string]interface{}{<br>            &quot;text&quot;: &quot;cc&quot;,<br>            &quot;someField&quot;: &quot;dd&quot;,<br>        },<br>    },<br>}</pre><pre>jstr, _ := json.Marshal(obj)</pre><p>But this is the price of real strong typing.</p><h4>Network protocol manipulation:</h4><p><strong>JS</strong> has all the standard mechanisms for creating servers &amp; proxies so in most cases like REST services, Websockets &amp; your run of the mill proxies this will suffice, but when required to alter low level http server calls, or create some special protocol adaptations, the base lib doesn’t allow for enough control, and you eventually resort to hacks &amp; workarounds.</p><p><strong>Go</strong> is a natural choice for deep protocol programming, since it allows for extreme flexibility. In http handling it provides a way to hijack the underlying TCP channel, it’s simple io.reader / writer interfaces allow for full control, without the abstractions of streams. My only gripe with the mechanism is that for some reason the http base library uses buffered io directly, instead of just implementing io.reader.</p><h4>Native integrations:</h4><p><em>JavaScript/TS </em>won’t get you far in this category, usually just invoke some binary executable that you have packaged along with the module in several OS versions, there is always Gyp, but i would never recommend it.</p><p><em>Go </em>has a several mechanisms, including:</p><ul><li>Unsafe Go code</li><li>CGo: writing c inside comments which is then compiled (can also include cpp or ObjectiveC syntax &amp; files)</li><li>The “syscall” module is built into the language and is handy when loading and invoking windows dlls.</li></ul><p>for writing cross-platforming edge cases with specifically built native code per OS I would prefer Golang, but if you ask me, most of the time I would opt to have cmdline integrations (a small exe to do the job) and stick with 100% Cross-OS code for most of my solution and 100% native code for some small satellite binary for some few native operations (not talking about mobile here which is a different matter completely).</p><h4>Error handling:</h4><p><strong>Java/C#/Python</strong>: Error handling by Exceptions, in these languages most errors, down to the base libraries are treated as exceptions including trivial stuff like “file not found”, Java is the worst by far, forcing the programmer to declare which exceptions can be thrown, which begs the question: “if you expect them, why are they called <strong>exceptions</strong>?”…</p><p>Error handling in <strong>JS/TS</strong> depends on whether you are using promises, if not then the error is usually returned as a callback parameter, for promise errors they are similar enough to exception handling, that we can say that errors are usually thrown trough the application unless explicitly addressed. Also, like most previous languages, null or undefined values are prone to create the notorious null pointer errors which usually account for 90% of code errors.</p><p>In Golang errors are a bit different, although the exception mechanism exists (called panic), it is not encouraged. Framework libraries will seldomly panic, and never on routine stuff, instead there is an <strong>err</strong> return value which requires your attention, even if only to ignore it by using the underscore “_”.</p><p>Null pointer errors are reduced by passing the receiver object to the function (receiver = the “<strong>this” object </strong>in most languages, and sometimes in JS too), so null checks can be done on the receiver before running the function, this means that foo.RunMe() doesn’t have to throw a “foo is undefined” error automatically.</p><h4>Code cleanliness:</h4><p>Most of the languages we are discussing are C syntax, and being so are somewhat similar, JS, TS, Java, C# syntax feels the same when coding.<br>Python and Go both break from the pack, with python striving for better readability, and Go for a simplified language and better performance.</p><p>Go is a very thought out language, but it sometimes enforces longer code in basic cases, getting in the way of fluent code writing:</p><ul><li>Bitty operations (no triary operator in Go)</li></ul><pre><strong>JS/TS (+Java/C#): </strong><br>res = (aa%2 == 0)?&quot;even&quot;:&quot;odd&quot;;</pre><pre><strong>Py3:<br></strong>res = &quot;even&quot; if (aa%2 == 0)  else &quot;odd&quot;</pre><pre><strong>Go:</strong><br>var res string<br>if aa%2 == 0 {<br>    res = &quot;even&quot;<br>}<br>res = &quot;odd&quot;</pre><ul><li>Basic String manipulation</li></ul><pre><strong>JS/TS (similarly Java/C#):<br></strong>(&quot;stUff&quot;+1).toLowerCase().startsWith(&quot;stu&quot;);</pre><pre><strong>Py3:<br></strong>(&quot;stUff&quot;+str(1)).lower().startswith(&quot;stu&quot;)</pre><pre><strong>Go:<br></strong>strings.HasPrefix(strings.ToLower(&quot;stUff&quot;)+strconv.Itoa(1),&quot;stu&quot;)</pre><ul><li>Append to array</li></ul><pre><strong>JS/TS:<br></strong>arr.push(&quot;Hola&quot;);</pre><pre><strong>Py3</strong>:<br>arr.append(&quot;Hola&quot;);</pre><pre><strong>Go:<br></strong>arr=append(arr,&quot;Hola&quot;)</pre><h3><strong>Ecosystem (Tooling, DevOps &amp; Community)</strong>:</h3><h4><strong>Modules &amp; Dep management</strong></h4><p><strong>Java</strong>: Maven/Gradel: good dep management, but takes very long to download and really hard to understand when it fails.</p><p><strong>C#</strong>: Nuget is ok, but it is definitely not as rich as other community component ecosystems.</p><p><strong>Python</strong>: Pip is much better than most, but the confusion between Py 2.7 and 3, and the fact that most people use a virtual-env to avoid collisions, shows it is less then perfect.</p><p><strong>TS/JS:</strong> In my opinion NPM is currently the best package manager &amp; module repository in the market, it handles deep dependency trees, multiple dependency versions, semantic versioning and shrink-wrapping for deep dependency version stability, as well as having solutions for local dep-repositories and global dep proxying. This is both a blessing and a curse since it created an ecosystem where importing a single dependency can drag 50 or more sub dependencies onto your project.</p><p><strong>Golang</strong> has a very fresh approach to dependencies, it uses git repos to get all dependencies as code, and compiles everything at the end, this approach solves several typical issues for compiled languages: dependency linking and OS Specific binary formats. But when going on to building enterprise grade projects it also creates problems with module versioning, dependency repetition with different versions, etc. There have been a ton of different community projects trying to define the right way to handle this and several official ones. Currently there are 2 official tools plus one in the making:</p><ul><li><strong>go get</strong>: the built in tool to get dependencies, has no solution to versioning doesn’t really work with tfs/stash/other non-github git servers, (but if you really want to make it work there is a <a href="http://ppanyukov.github.io/2017/02/01/golang-with-vsts-repos.html">hack</a>)</li><li><a href="https://github.com/golang/dep"><strong>dep</strong></a>: self defined as an “official experiment” for go 1.9+, and offers some solutions to the problem by keeping a file that maps the versions your code imports and locks them to specific commits in the respective git repos.</li><li><a href="https://github.com/golang/go/wiki/vgo#current-state"><strong>VGO</strong></a><strong> / go modules</strong>: an experimental dependency management, different in approach from dep, which is now tentatively adopted for go 1.11, I took it for a spin, and it seems much better than prev methods, allowing for projects to be outside of the gopath, and working trough “go get” I hope it is finalized in Go 1.12, so we will finally have a good dep management built in.</li></ul><p>In short, Golang’s CI/CD story is still ramping up, and any toolchain you adopt will likely be modified in the near future, but we should keep in mind that this was also the case for many typescript facilities (e.g. typings) just a couple of years ago.</p><h4><strong>Deployment, Binary Size, Signing, Obfuscation</strong></h4><p>NodeJS binaries are just a seaming heap of JS code, including all the dependencies you have recursively imported. it usually gets to ~300Mb and more without any trouble for a medium sized project.</p><p>This is where Go really shines:</p><ul><li>Binaries are self contained and OS Specific everything is portable, just download and run.</li><li>Native binaries can be signed and the signature would be verified by the OS when running them.</li><li>Binaries are hard to understand, even when compared to cpp binaries, since they have the addition of the garbage collection mechanism to confuse IDA debugging / cracking.</li></ul><h4>Debugging / IDE Environments</h4><p>We are all used to very advanced debugger environments carefully tuned by professionals, this goes for JS / TS, as well as C#, Java and most other languages I wrote, <strong>C#</strong> having the best of microsoft support in VsCode &amp; VStudio with shiny features like Edit and continue and support for moving instruction pointer while debugging, as well as advanced refactoring support with jetbrains resharper. <strong>Java</strong> obviously having the most IDEs around, and <strong>JavaScript</strong> having some of the best support as well with both Jetbrains support and VsCode, as well as various editors and in-browser debugging. <strong>Typescript</strong> debugging is supported where JS is supported with map files, and it also has good VsCode support and IntelliJ/WebStorm support.</p><p><strong>Python </strong>doesn’t have many IDE choices, and many use advanced text editors, it does have Good VsCode support and PyCharm/IntelliJ, but it doesn’t have as much options or as deep an integration as some of the other languages here.</p><p><strong>Golang</strong> debugging is not yet perfect. The delve debugger has only reached 1.0 status <a href="https://github.com/derekparker/delve/releases/tag/v1.0.0">around 6+ months ago</a>, and it is not yet as fully featured and tuned. The IDE environments are also catching up: <strong>VSCode </strong>has basic Go support, but it runs into trouble doing basic refactoring like renaming variables, has no function call support in console or watch, no new breakpoints set while debugging, and a limited visibility into variable reference trees and large arrays. <strong>Goland / IntelliJ </strong>plugin is better and is said to have google’s support, fast refactoring, quicker response times and better variable visibility, but it is a payed option.</p><h3>The bottom line:</h3><p>While C# and Java are obviously fueled by legacy-code driven inertia, TS, Py and Go are here for a reason, NodeJS/TS wins in categories like: “<strong>Fastest time to market</strong>” and “<strong>Best tool-chain</strong>”, Python wins “<strong>Easiest collection manipulation and number crunching</strong>”, which explains its popularity in DataSciense, and Golang would take: “<strong>Best memory consumption</strong>”, “<strong>Stable under pressure</strong>” and “<strong>Easiest to deploy on premise</strong>”. Golang is a more powerful platform than NodeJS, allowing more abilities to tamper with protocol channels, and to wield the might of parallel computing in a comparably streamlined way, most advanced programmers will enjoy working with it, although an average Dev might prefer JS.</p><p>As for me, Golang takes me back to the time when “one exe” was a worthy goal and the most professional software was extremely portable, tightly packaged and tamper proofed. I love it for that… somehow I still think there are types of software that should come out of the factory with a shiny hard shell, and not as a pile of a thousand loosely connected parts that function well when assembled correctly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d751eb386184" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/language-awards-war-of-codes-d751eb386184">Language awards: war of codes</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Home Volume Control: Go+VueJS = R&R]]></title>
            <link>https://itnext.io/home-volume-control-go-vuejs-r-r-151ff030ffa8?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/151ff030ffa8</guid>
            <category><![CDATA[vuejs]]></category>
            <category><![CDATA[simplicity]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[service-discovery]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Sun, 05 Aug 2018 06:40:31 GMT</pubDate>
            <atom:updated>2018-08-05T07:49:02.016Z</atom:updated>
            <content:encoded><![CDATA[<p>A self-discovering no-config, home volume control system in Golang &amp; Vue.Js</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8aM9MDF7uHppmcHnr8_q2w.jpeg" /></figure><p>In most homes today we have some home computer for the kids, some laptops, and several tablets/phones. Now just try to get them all to press the volume-down buttons when you want to rest… How about a self-discovering, distributed service for controlling volume around the house?… Did I hear someone saying: “Give us some snazzy UI to remote control our kids with”? your wish is my command.</p><p>Every now and then, I get an itch to write a code project, which would act as a testing ground for new tech, and expose me to problems and ideas I wouldn’t otherwise have explored. I really like Golang’s ability to present a small, self contained service, which includes a REST backend &amp; Web-server in one smallish exe file (and a public dir for html/js), I also really liked VueJS for the same kind of simplicity, It has no problem having just one html file for everything. I wanted to test this notion by building a small home project, I also got to explore network service-discovery, just to spice it up. I might later take this experiment into <a href="https://github.com/golang/go/wiki/Mobile">go-mobile</a> or progressive web apps.</p><p>The complete source code for this project can be found on <a href="https://github.com/amitbet/kid-control">my github</a> page.</p><h4>Service discovery options:</h4><ul><li>SSDP: <a href="https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol">simple service discovery protocol</a> (<a href="https://github.com/koron/go-ssdp">go-ssdp</a>)</li><li>Zeroconf: uses <a href="https://en.wikipedia.org/wiki/Multicast_DNS">multicast DNS</a> (golang <a href="https://github.com/grandcat/zeroconf">zeroconf</a>)</li></ul><p><strong>SSDP</strong>: one of the original “simple” protocols that was invented along with the internet itself, based on UDP multicast, very easy to implement and incorporated into upnp, but rather overused and might be blocked by home router configuration. The go implementation only had 15 stars, hmm..</p><p><strong>ZeroConf</strong>: zconf uses Multicast DNS, <a href="https://github.com/grandcat/zeroconf">the go implementation</a> has 140 stars, <a href="https://github.com/hashicorp/mdns">go-mdns</a> (hashicorp) has 450 , this sounds more like mainstream code to me. mDNS is also based on UDP multicast (bi-directional this time), it is said to be more resilient to common home network mis-configurations. Both are equally non-secure, but we are talking about home networks...</p><p>I would have gone with zeroconf, but after playing with it (<a href="https://sosedoff.com/2017/09/07/zeroconf.html">guide</a>), it seems to require <a href="https://stackoverflow.com/questions/23624525/standard-mdns-service-on-windows">installing a Bonjour client</a>, or applying some reg hack to disable a service that is listening on the mDNS default ports. I was naively thinking that the whole idea of zeroconf was ZERO configuration. So I ended up opting for some <strong>SSDP</strong> <strong>Simplicity</strong>.</p><pre>import (<br>    ssdp &quot;github.com/koron/go-ssdp&quot;<br>)</pre><pre>hname, err := os.Hostname()</pre><pre>//advertizing the service<br>ad, err := ssdp.Advertise(<br>  &quot;urn:schemas-upnp-org:service:KidControl:1&quot;, // send as &quot;ST&quot;<br>  &quot;id:&quot;+hname,                                 // send as &quot;USN&quot;<br>  &quot;http://&quot;+myIp+&quot;:7777/&quot;,                     // send as &quot;LOCATION&quot;<br>  &quot;ssdp for KidControl&quot;,                       // send as &quot;SERVER&quot;<br>  3600) // send as &quot;maxAge&quot; in &quot;CACHE-CONTROL&quot;<br> if err != nil {<br>  logger.Error(&quot;Error advertising ssdp: &quot;, err)<br> }</pre><pre>//searching for instances on the network<br>svcList,err := ssdp.Search(&quot;urn:schemas-upnp-org:service:KidControl:1&quot;, 5, &quot;&quot;)</pre><p>After settling that debate, let’s go on to writing the actual service. we’ll use go-volume to control volume Cross-OS, on windows we should make sure we are not in a remote session (that doesn’t work). Laying down some boilerplate server code, for get/set vol REST API, and serving up a public web root:</p><pre>import (<br>    gmux &quot;github.com/gorilla/mux&quot;<br>)<br><br>mux := gmux.NewRouter()</pre><pre><strong>//restfull API:</strong><br>mux.HandleFunc(&quot;/set-volume&quot;, setVolumeOnMachine).Queries(&quot;machine&quot;, &quot;{machine}&quot;)<br>mux.HandleFunc(&quot;/set-volume&quot;, setVolume)<br>mux.HandleFunc(&quot;/get-volume&quot;, getVolumeOnMachine).Queries(&quot;machine&quot;, &quot;{machine}&quot;)<br>mux.HandleFunc(&quot;/get-volume&quot;, getVolume)<br>mux.HandleFunc(&quot;/configuration&quot;, sendConfig)</pre><pre><strong>//web server:</strong><br>mux.PathPrefix(&quot;/&quot;).Handler(http.FileServer(http.Dir(&quot;./public&quot;)))</pre><pre><strong>//run the Http listener:</strong><br>err := http.ListenAndServe(&quot;:8080&quot;, mux)<br>if err != nil {<br>    fmt.Print(&quot;lisgtening error: &quot;, err)<br>}</pre><p>Now on to the JS side for a nice web UI. Since I am not a web designer, and have no aspiration to be one, I’ll just pickup some nice css suite (e.g.: bootstrap) to take care of responsive tables and rounded buttons for me.</p><pre>&lt;link rel=&quot;stylesheet&quot; href=&quot;https://stackpath.bootstrapcdn.com/bootstrap/4.1.0/css/bootstrap.min.css&quot; &gt;</pre><h4>The old Vue / Angular / React debate:</h4><p>Since we will have a dynamic repeating part, for controlling the volume in each discovered machine, we need some framework to generate dynamic HTML tags and bind them to the data we get from the server.</p><p>This is where <strong>VueJS</strong> comes in, now you say: “It could have been any other JS framework why pick vue? why not Angular v1.5/v6, ReactJS etc…” Well, these other frameworks normally require a considerable installation, along with some UI build process and some time spent learning the specifics of their not-so-intuitive operation. Digging around the network for minimal angular/react demos I did find some, but they are either <a href="https://github.com/dmitriz/min-angular/blob/master/demo/index.html">old</a> or <a href="https://medium.com/@to_pe/how-to-add-react-to-a-simple-html-file-a11511c0235f">hacky</a>, and not recommended for production. Vue is extremely easy to integrate, just like bootstrap, you drop a “s<em>cript src” </em>link into the project and you are ready to start coding with Vue without a need for a build chain (no webpack, pre-compilation steps, plugins, etc.) and this is official and recommended on their <a href="https://vuejs.org/v2/guide/index.html">main page</a>. “OK then, this Vue thing sounds like good fun. Sign me up!”</p><p><strong>Let there be Vue:</strong></p><pre>&lt;script src=&quot;https://unpkg.com/vue&quot;&gt;&lt;/script&gt;</pre><pre>&lt;div class=&quot;form-group m-3&quot; id=&quot;vueApp&quot;&gt;<br>&lt;h3&gt;Change Volume&lt;/h3&gt;</pre><pre><strong>&lt;!--we start using vue, and attach it to the &#39;vueApp&#39; ID later--&gt;</strong><br>&lt;div <strong>v-for=&quot;m in machines&quot;</strong> class=&quot;row&quot;&gt;<br>&lt;div class=&quot;col-md-1&quot;&gt;<br>    &lt;b&gt;<strong>{{m.name}}</strong>&lt;/b&gt;<br>&lt;/div&gt;<br>&lt;div class=&quot;col-md-2&quot;&gt;<br>    &lt;input type=&quot;range&quot; <strong>v-on:change</strong>=&quot;changeVolume(m.volume,m.name)&quot; min=&quot;0&quot; max=&quot;100&quot; <strong>v-model</strong>=&quot;m.volume&quot; id=&quot;volumeSlider&quot;&gt;&lt;/input&gt;<br>&lt;/div&gt;<br>&lt;div class=&quot;col-md-1&quot;&gt;<br>    &lt;input type=&quot;text&quot; <strong>v-on:change</strong>=&quot;changeVolume(m.volume,m.name)&quot; id=&quot;volumeText&quot; <strong>v-model</strong>=&quot;m.volume&quot;&gt;&lt;/input&gt;<br>&lt;/div&gt;<br>&lt;/div&gt;</pre><p>Some JQuery ajax functions to connect VueJS to our REST API:</p><pre>&lt;script src=&quot;https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js&quot;&gt;&lt;/script&gt;</pre><pre>&lt;script&gt;<br>        <strong>//the JS part of vue, this is where we attach to the tags</strong><br>        var appData = new Vue({<br>            el: <strong>&#39;#vueApp&#39;</strong>,<br>            data: {<br>                machines: [{<br>                        name: &quot;localhost&quot;,<br>                        volume: 100<br>                    },<br>                ]<br>            },<br>            mounted: function () {<br>               <strong>//the &quot;on document ready&quot; function</strong><br>                var self = this;<br>                $.get(&quot;http://&quot; + window.location.host + &quot;/configuration&quot;, function (data) {<br>                    let d = JSON.parse(data)<br>                    self.machines = d.machines;<br>                });<br>            },<br>            methods: {<br>                <strong>changeVolume</strong>: function (slideValue, machine) {<br>                    let url = &quot;http://&quot; + window.location.host + &quot;/set-volume?machine=&quot; + machine</pre><pre>$.ajax({<br>                        url: url,<br>                        data: JSON.stringify({<br>                            &quot;volume&quot;: slideValue,<br>                        }),<br>                        dataType: &#39;json&#39;,<br>                        type: &#39;POST&#39;,<br>                        contentType: &#39;application/json&#39;,<br>                        success: response =&gt; console.log(response),<br>                        error: e =&gt; console.log(e)<br>                    });<br>                }<br>            }<br>        })</pre><pre>&lt;/script&gt;</pre><p>Presto! Go around running it on a number of computers around the house (Win/Linux/Mac), afterwards connecting to any one of them will give you something like this display:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5CEq9LU4hUF3LggSXT-lEw.png" /><figcaption>Playing around with the volume around the house was never so easy!!</figcaption></figure><p>Now there is a shiny new webapp to shush the kid’s youtube streams with. It was also a ton of fun writing the application… I love simplicity when it comes to code, I think most programmers do, it is one of the reasons Node, JSON and JS are so successful. There is something magical about starting up easy, gaining complexity only when asked for, which makes it all seem like a kids construction game. In my opinion, frameworks with steep learning curves are very rarely justified, this drive towards simplicity is why developers keep rebuilding applications and frameworks from scratch, always thinking: “there has to be an easier way to accomplish the task”.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=151ff030ffa8" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/home-volume-control-go-vuejs-r-r-151ff030ffa8">Home Volume Control: Go+VueJS = R&amp;R</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Golang bits: Better Debugging in VSCode]]></title>
            <link>https://itnext.io/golang-bits-better-debugging-in-vscode-599bc5b018da?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/599bc5b018da</guid>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[tips]]></category>
            <category><![CDATA[refactoring]]></category>
            <category><![CDATA[vscode]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Sun, 15 Jul 2018 07:40:01 GMT</pubDate>
            <atom:updated>2018-07-24T10:34:46.106Z</atom:updated>
            <content:encoded><![CDATA[<p>VSCode is great! but for golang renaming &amp; debug is just …meh?, read on!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oh2XiV21MwxNDl7yb7imyA.png" /></figure><h3>Debugger — Max string size &amp; inspect nesting limit:</h3><p>The golang debugging experience in VSCode is poor since viewing deeply nested variables is cutoff when you reach the 3rd level, and viewing strings is lmited to 64 characters which is almost never enough, the same goes for arrays, however, that bothers me less.</p><p>The vscode team has added support for delve v2 APIs, but these are still turned off by default in the current version (v1.25.0). To turn it on, and control these pesky variables, change the vscode user settings to stop using delve v1 API, and control the MaxStrLen and MaxVeriableRecurse variables like this:</p><p>file -&gt; preferences -&gt;settings</p><pre>&quot;go.delveConfig&quot;: {<br>  &quot;useApiV1&quot;: <strong>false</strong>,<br>  &quot;dlvLoadConfig&quot;: {<br>    &quot;followPointers&quot;: true,<br>    &quot;maxVariableRecurse&quot;: <strong>3</strong>,<br>    &quot;maxStringLen&quot;: <strong>400</strong>,<br>    &quot;maxArrayValues&quot;: <strong>400</strong>,<br>    &quot;maxStructFields&quot;: -1<br>  }<br>}</pre><p>*just remember that the “maxVariableRecurse” <strong>will </strong>slow down your debugging, so play with it to see what is acceptable to you.</p><h3>Slow debug launch:</h3><p>Shut down windows defender antivirus real time protection:</p><p>start menu -&gt; search for “virus” , click on “virus &amp; threat protection” -&gt; click on “virus &amp; threat protection settings” -&gt; turn off “real time protection”</p><p>On my windows 10 machine, It took very long to launch when pressing F5, so much so, that I could wait something around 40 seconds each time I would start a debugging session, turning the real time protection off made it sane again.</p><h3>Slow renaming:</h3><p>vscode uses golang’s own rename tool to do the job, but this tool searches trough your entire go path, which can be a ton of usually irrelevant code. This gorename fork by prateek, disables global search and makes renaming</p><pre>go get github.com/prateek/gorename</pre><p>This should compile gorename from source, now you should test that it is installed properly in your path</p><pre>(on linux/mac:) which gorename<br>(on windows:) where gorename</pre><p>also check that the gorename creation date which should be brand new.</p><h3>JSON is a pain to fomalize in golang</h3><p>Go is a very easy language to write in the most part, but formalizing JSON into types is quite tedious when compared to typescript, the “paste to json” vscode addon uses an open source called <a href="https://github.com/quicktype/quicktype">quicktype</a> to create pretty good types for a given JSON sample, it even finds similar types creating no duplicates, if you are on the strict side you may just need to cosmetically rename a couple of inner types at the end, but now that the rename works fine, you can..`</p><ul><li>Install “Paste JSON as Code”</li><li>Reload</li><li>Create a new .go file</li><li>Copy your json</li><li>Press: ctrl + shift + p</li><li>Type “paste json as code”, press enter</li><li>When prompted, set a name for the root object</li><li>Add <em>,omitempty</em> where needed</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=599bc5b018da" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/golang-bits-better-debugging-in-vscode-599bc5b018da">Golang bits: Better Debugging in VSCode</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Defining the role of System Architect]]></title>
            <link>https://medium.com/@amit.bezalel/defining-the-role-of-system-architect-7461c151ab1d?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/7461c151ab1d</guid>
            <category><![CDATA[workplace]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[innovation]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Fri, 15 Jun 2018 21:21:15 GMT</pubDate>
            <atom:updated>2019-06-28T03:03:05.152Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KCXOk0yOz28ow21knVlueg.jpeg" /></figure><p>Unlike most IT company positions, an Architect’s position is both more liberated and more ambiguous, here are my thoughts about the job.</p><p>Given I’m a serial System Architect (I’ve been an architect for 3 different large-scale software projects in two unrelated companies) I feel I am in a unique position to share my thoughts about the actual meaning of this role.</p><p>Some online course I saw once, said any group will require several roles: <strong>Organizer, Leader, Spokesman, Quality inspector</strong>, and can also benefit from: <strong>An Innovator, An Energizer and A Harmonizer</strong>. It is safe to say that managers can fill most of the first set of requirements, but will usually not have time, technical skills or the right position (due to hierarchical distance) to fulfill the second set. That is where Architects come into play. The optional nature of these roles may also explain the late emergence of the architect’s position in the IT job market.</p><p>Let’s take a moment to explain these “extra” roles: An <strong>Energizer</strong> adds drive and motivation and keeps a positive vibe throughout the group, a <strong>Harmonizer</strong> keeps subgroups and individuals in sync as well as integrating with external groups, airing out and fixing disagreements. the <strong>Innovator</strong> will suggest new ideas, show the group new ways of getting the job done, and prove the worthiness of investing in new implementation paths and technologies, the innovator is the scout that leads the way by exploring next year’s features.</p><p>An additional role the architect holds is <strong>Technical decision making </strong>which may also swing widely between authority and advice, depending on the architect’s own style and the workplace. Time constraints and managerial decisions may ultimately have the last say about specific features, but an architect must warn against bad shortcuts and also keep a watchful eye for any “wayward implementations” which happen when developers don’t understand the importance of the architecture or feel stressed to complete the task quickly.</p><p>As the range of skills that make up the architect’s job are wide, one may see why it escapes precise definition, creating many possibilities for leading and leaving your mark on things, and as many “opportunities” to get completely lost in the details.</p><h4>About Good software design</h4><p>Whether you are building Microservices or just a large desktop/phone app, your system will have a box chart design, that shows the different components their connections. <strong>Assigning clear responsibilities to each box</strong> makes reasoning about which code goes where much easier, e.g. by saying “user management is done by X alone”. As the system grows new features will create new responsibilities which get assigned to components either by continuous choice (architecture), or by ongoing work (coded anywhere and everywhere). Make sure you know about new responsibilities, add them to existing components or create new ones. Another part of this important process is defining component interactions, here protocols and schema are defined, and versioning applied to give clarity to supported features of each component. This information should be passed on to the team, not as a one time deal, but as a living model that you should debug against together while solving code &amp; design problems. <strong>Architecture is a living entity just like the system itself</strong>, <strong>and should be treated as such</strong>. Charts can also be coded and committed to source control as plantText documents.</p><h4>Becoming a software architect</h4><p>Several programmers asked me how does one become an architect, my answer is that like with most positions, displaying the qualities and fulfilling the roles will eventually gain you the title. Any team mate which will start coming up with innovations, presenting about new tech, writing blogs, and influencing design decisions will get noticed and marked as a thought leader. Now that you know that an architect should also be a harmonizer, you can add passing information &amp; promoting inter-team collaboration to this list. Start your way by rotating lunch groups, get to know different people from many teams and expand your network.</p><h4>Walking the code line:</h4><p>I have met architects with different types and styles, some of them leaning towards the technical and mostly ignoring the social aspects of the job, repeating the geek’s mantra: “If I wanted to be a people’s person I wouldn’t have chosen talking to machines as a career”, thus loosing much of their influence inside the group. Others lean away from the technical side thinking: “i’m on the design level now…”, thus lose their connection to the actual code, and slowly turning into “box drawing” designers. An architect needs the abilities to review existing code, choose new technologies and inspire his developers, so he needs to stay “hands on” totally connected to the implementation, and occasionally write some part of it himself.</p><p>Additional dangers are over investing in innovation, or in planning. Contact with real world problems is the root of all inventions,<em> innovation should come as an answer to someone’s bothering itch</em>, it should also match the needs of the specific market. Disconnected innovations may be viewed as odd novelties. Planning too far and wide can lead to producing grand, over engineered designs which entail programming and usage complexity. Such complex, hard to understand designs eventually make people shy away from the project.</p><h4>My take on things</h4><p>As I see it, the architect should be a people’s person which is in sync with the group, a structure planner, finding the right paths to take and, most importantly, an experienced, hands on programmer which is actively contributing code and guiding other’s implementations. Above all an architect <strong>has to keep contributing</strong>, since writing code and talking to programmers daily, keeps him <strong>connected to real world problems.</strong> An architect that does not offer relevant solutions to today’s and tomorrow’s problems is just another person with an impressive ability to talk about black boxes on whiteboards.</p><p>Despite including undefined responsibilities and hidden pitfalls, the architect’s job is actually a unique position with a huge potential for impact on any software project, I will leave you with a quote from a fellow architect, which stated: ”A system’s architecture tends to last much longer than was expected at the time of it’s conception, sometimes even outlasting it’s original code”. I would add that a good architect also has a huge impact on how much <strong><em>Fun</em></strong> it is to work on a software project, a fact that most people neglect to consider.</p><p>For further reading about the software architect position, see <a href="https://www.toptal.com/software-architect#hiring-guide">this article</a> from my friends at Toptal.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7461c151ab1d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[State of the Market: Voice dictation APIs]]></title>
            <link>https://medium.com/@amit.bezalel/state-of-the-market-voice-dictation-apis-2f7d09c346d5?source=rss-f2d02b8804e0------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f7d09c346d5</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[dictation]]></category>
            <category><![CDATA[deep-speech]]></category>
            <category><![CDATA[voice-recognition]]></category>
            <category><![CDATA[text-to-speech]]></category>
            <dc:creator><![CDATA[amit bezalel]]></dc:creator>
            <pubDate>Wed, 06 Jun 2018 12:29:21 GMT</pubDate>
            <atom:updated>2018-06-17T05:52:04.843Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vfoi9PuWUlJbqXw7aFqy9Q.png" /></figure><p>Transcribe-bot monster meltdown: DeepSpeech, Dragon, Google, IBM, MS, and more!</p><p>Speech has been a near-impossible field for computers until recently, and as talking to my computer has been something I dreamed of as a kid, I have been tracking the field as it progressed trough the years.</p><ul><li>Expectations from dictation are high, correcting more than two words per paragraph means to most people it is just easier to type the whole thing in.</li><li>Microphone quality is very important to speech transcription, <br>$50 headsets are just “ok”, for best results the professionals use pro mics which cost $300+.</li><li>According to google Audio for recognition should <strong>not</strong> be compressed in a lossy way, as it hampers recognition. This means that network traffic might be an issue, and should be considered.<br>Calculating bandwidth requirements for <strong>16bit mono linear PCM</strong>:<br>48Khz = 96KB/sec (most microphones provide)<br>44.1Khz = 88KB/sec (default windows setting)<br>16Khz = 32KB/sec (lowest quality allowed)<br>Assuming ~100 concurrent streams: between 9.6MB and 3.2MB upload bandwidth should be allocated to this <strong>on the customer’s site</strong> for cloud based recognition. Flac can be used for lossless compression in order to save bandwidth, but that would require a Flac encoder.</li><li>Most cloud APIs will include a 60sec limit on streaming voice recognition:<br><strong>Wit</strong>: using chunked http upload, <a href="https://wit.ai/docs/http/20170307#streaming-audio">10sec limit<br></a><strong>IBM: </strong>allows to configure the time limit (also allowing to remove it)<strong><br>Apple</strong>: 60sec limit, unknown protocol masked by iOS SDK<br><strong>Google</strong>: using grpc over http2, <a href="https://cloud.google.com/speech-to-text/docs/basics">65sec limit</a> on streaming recognition.<br><strong>Microsoft</strong>: using websockets, <a href="https://docs.microsoft.com/en-us/azure/cognitive-services/speech/api-reference-rest/websocketprotocol">10 minute limit</a><br>These limitations can usually be worked around by reconnecting periodically to the service.</li></ul><h3>The opensource &amp; public cloud market:</h3><p><strong>The list is ordered approximately by recognition quality &amp; fit for cloud streaming dictation:</strong></p><ul><li><a href="https://cloud.google.com/speech-to-text/"><strong>Google Speech API</strong></a><strong>: [2.4</strong>¢/<strong>min] </strong>[<a href="https://cloud.google.com/speech-to-text/"><strong><em>Demo</em></strong></a>] wins the match in most blogs I read, also gets the popular vote, you can try It online &amp; on google docs (tools =&gt; voice typing), Google also have some better models under development and in alpha stages, which are available to Google partners .</li><li><a href="https://www.ibm.com/watson/services/speech-to-text/"><strong>IBM watson</strong></a><strong>: [2</strong>¢/<strong>min]</strong>[<a href="https://speech-to-text-demo.ng.bluemix.net/?cm_mc_uid=32254540268215275786482&amp;cm_mc_sid_50200000=35225731527578648261&amp;cm_mc_sid_52640000=78471271527578648267"><strong><em>Demo</em></strong></a>]<strong> </strong>looks like a well rounded speech recognition engine which is used in the IBM Watson project, offered as a part of the IBM bluemix cloud.</li><li><a href="https://azure.microsoft.com/en-us/services/cognitive-services/speech/"><strong>Microsoft Bing Speech</strong></a><strong>: [~2</strong>¢/<strong>min]</strong>[<a href="https://azure.microsoft.com/en-us/services/cognitive-services/speech/"><strong><em>Demo</em></strong></a>]<strong> </strong>try online just click the link (many languages), try in office with <a href="https://dictate.ms/">dictate.ms</a>, A Microsoft garage product</li><li><a href="https://developer.apple.com/documentation/speech?changes=_8"><strong>Apple’s Speech API</strong></a><strong>: [free for iOS apps] </strong>looks better than Microsoft in most benchmarks, may only be accessed from iOS &amp; MacOS devices (many languages)</li><li><a href="https://medium.com/wit-ai/introducing-wit-speech-api-6c3a514cf245"><strong>Wit Speech</strong></a><strong>:</strong> [<a href="https://labs.wit.ai/demo/index.html"><strong><em>Demo</em></strong></a>] less info on this engine as it is a smaller company, it is geared towards voice assistants &amp; commands.</li><li><a href="http://kaldi-asr.org/"><strong>Kaldi</strong></a><strong>: [Free OpenSrc] [</strong><a href="https://github.com/amitbet/speech/tree/master/kaldi"><strong>dockerfile</strong></a><strong>, </strong><a href="https://hub.docker.com/r/amitbet/kaldi/"><strong>docker</strong></a><strong>] </strong>The most mature speech recognition open source, has streaming recognition via <a href="https://github.com/alumae/kaldi-gstreamer-server">gstreamer server</a>, I don’t expect it to compare to google, but is an option for on premise engines, plus it’s free… (English &amp; Estonian only).</li><li><a href="https://techcrunch.com/2017/11/29/amazon-transcribe-is-a-sophisticated-transcribing-service-for-aws/"><strong>Amazon transcribe</strong></a>: <strong>[2.4</strong>¢/<strong>min] </strong>no streaming recognition yet, upload only for now. I don’t currently have access to this service, so feel free to use my <a href="https://github.com/amitbet/speech/tree/master/testing-audio">audio files</a> and send me the results. (many languages)</li><li><a href="https://www.temi.com/"><strong>Temi</strong></a><strong> / HappyScribe / Trint / Spext</strong>: <strong>[~10</strong>¢/<strong>min] </strong>Dedicated voice transcription companies which have a cloud transcribing offering. streaming / real-time recognition is not their focus, as they are geared towards offline, long form audio transcriptions. but it is worth sampling in order to compare accuracy across the market, so I chose to run the samples trough Temi as well.</li><li><strong>Mozilla </strong><a href="https://blog.mozilla.org/blog/2017/11/29/announcing-the-initial-release-of-mozillas-open-source-speech-recognition-model-and-voice-dataset/"><strong>Deep Speech</strong></a><strong>: [Free OpenSrc] [</strong><a href="https://github.com/amitbet/speech/blob/master/deepspeech/Dockerfile"><strong>dockerfile</strong></a><strong>, </strong><a href="https://hub.docker.com/r/amitbet/deepspeech/"><strong>docker</strong></a><strong>] </strong>one of the most recent and most exciting speech engines, created by implementing a Nueral network designed by Andrew Ng on tensorflow, and with Mozilla’s Data Sciense team actively behind it, It looks like a very promising OpenSource offering. It still has no streaming recognition: <a href="https://github.com/mozilla/DeepSpeech/pull/1275">coming soon</a>? (English Only for now)</li></ul><h3><strong>Professional cloud speech engines</strong></h3><p>These professional engines look better than google speech in most benchmarks I saw, they have “talk to us” OEM typed licensing, and have Medical grade speech engines that are being actively used in the healthcare industry to save time while filling out patient reports. Needless to say that tolerance for errors in this field is very low.</p><ul><li><a href="https://www.nuance.com/dragon/dragon-anywhere.html"><strong>Nuance Dragon</strong></a><strong>: </strong>Nuance offers a range of solutions including desktop toolbars, on-premise speech recognition servers, and a cloud speech API, A free trial is offered to customers doing POCs, again, with a “talk to us first” policy. I used dragon pro 15, which is their non-medical dictation desktop app. (many languages)</li><li><a href="https://sayit.nvoq.com/"><strong>nVoq SayIt</strong></a><strong>:</strong> [<a href="https://eval.nvoq.com/webdictation/"><strong><em>Demo</em></strong></a>] best in KLAS for medical speech, powering Dolbey’s fusion cloud speech APIs. Trials on “talk to us” basis here as well. (North American English only)</li></ul><h3>Comparing the engines</h3><p>There already are a ton of comparisons trying out the different services side by side, and I will instead link to some prominent blogs at the end, but I couldn’t leave you all without a good experimentation run just for the fun of it.</p><h4>The test setup</h4><p>I recorded four short lines two of text from Darwin’s origin of species and two from Alice in wonderland (wikipedia) which range in recognition difficulty, in my own voice through a regular headset, Such one-liners are easy to send to all services since some have length limitations. After processing (has to be 16khz 16b Linear PCM Wav) I uploaded the same audio files to all services. since temi &amp; deepspeech do not understand punctuation commands, I removed “comma” and “period” from the resulting text where they were recorded (this was a bit favorable towards DS which returned “camma” and other stuff instead).</p><p>All speech engines that support file upload in the demo / trial, received the audio files for translation, since Wit.Ai &amp; nVoq didn’t provide this feature, I used a <a href="https://www.vb-audio.com/Cable/index.htm">VB-Audio</a> device driver to transfer audio from a player to a virtual mic, voice was played at 90% volume.</p><p>I do not have an AWS account or a mac computer, so I don’t have access to these engines, if you have it, please run the audio through it, and i’ll add your text to the comparison. Kaldi was tested with the Tedlium model.</p><h4>Alice quote I</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttp%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F453101787%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Famit-be%2Falice1&amp;image=http%3A%2F%2Fa1.sndcdn.com%2Fimages%2Ffb_placeholder.png%3F1527840666&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c3c607d4ff7715451308b80035d1952/href">https://medium.com/media/4c3c607d4ff7715451308b80035d1952/href</a></iframe><blockquote><strong>Origin</strong>: Alice a girl of seven years is feeling bored and drowsy while sitting on the riverbank with her elder sister</blockquote><ul><li><strong>DS</strong>: alice a girl of seven years is feeling <strong><em>boared an drouwsy wild</em> </strong>sitting on the <strong><em>riv bank wit</em> </strong>her elder sister</li><li><strong>Google</strong>: Alice a girl of 7 years is feeling bored and drowsy while sitting on the riverbank with her elder sister.</li><li><strong>MS</strong>: Alice a girl of 7 years is feeling bored and drowsy while sitting on the riverbank with her elder sister.</li><li><strong>IBM</strong>: Alice A girl of seven years is feeling bored and drowsy while sitting on the riverbank with her elder sister.</li><li><strong>Temi</strong>: Alice, a girl of seven years is feeling bored and drowsy while sitting on the riverbank with her elder sister.</li><li><strong>nVoq</strong>: <strong><em>Palace </em></strong>a girl of 7 years is feeling bored and drowsy while sitting on the <strong><em>roof ban quoted </em></strong>her elder sister</li><li><strong>Wit.AI</strong>: <strong><em>how is the</em></strong> girl of seven years is feeling bored and drowsy while sitting on the riverbank with her elder sister</li><li><strong>Kaldi (tedlium)</strong>: <strong><em>place until</em></strong> seven years is feeling bored <strong><em>in trials the</em></strong> while sitting on the riverbank with her elder sister.</li><li><strong>Dragon Pro 15</strong>: Alice, a girl of seven years is feeling bored and drowsy while sitting on the riverbank with her elder sister</li></ul><h4>Darwin quote I</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttp%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F453101811%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Famit-be%2Fdarwin1&amp;image=http%3A%2F%2Fa1.sndcdn.com%2Fimages%2Ffb_placeholder.png%3F1527840666&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/04c89905edaa0afc3a2a3fa21a9ca6c3/href">https://medium.com/media/04c89905edaa0afc3a2a3fa21a9ca6c3/href</a></iframe><blockquote><strong>Origin</strong>: In the wild animal, on the contrary, all its facilities and power being brought into full action for the necessities of existence.</blockquote><ul><li><strong>DS</strong>: in the wild animal on the contrary all its <strong>vasilithies </strong>and power being brought into full action for the <strong>necissities </strong>of existence</li><li><strong>Google</strong>: In the wild animal, on the contrary, all its facilities and power being brought into full action for the necessities of existence.</li><li><strong>MS:</strong> In the wild animal comma On the contrary comma all its facilities in power being brought into full action for the necessities of existence.</li><li><strong>IBM</strong>: In the wild animal, on the contrary, all its facilities and power being brought into full action for the necessities of existence.</li><li><strong>Temi</strong>: In the wild animal, on the contrary, all its facilities and power being brought into full action for the necessities of existence.</li><li><strong>nVoq: </strong>In the wild animal, on the contrary, all its facilities and power being brought into full action <strong><em>4</em></strong> the necessities of existence</li><li><strong>Wit.AI</strong>: in the wild animal on the contrary its facilities and power being brought into full action for the necessities of existence</li><li><strong>Kaldi (tedlium)</strong>: in the wild animal on the contrary now <strong><em>it’s a sad that these</em></strong> and power being brought into <strong><em>for </em></strong>action for the necessities of existence.</li><li><strong>Dragon Pro 15</strong>: In the wild animal, on the contrary, all its facilities and power being brought into full action for the necessities of existence</li></ul><p>** facilities = <strong>faculties</strong>, I mispronounced, the STT didn’t care…</p><h4>Darwin quote II</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttp%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F453101817%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Famit-be%2Fdarwin2&amp;image=http%3A%2F%2Fa1.sndcdn.com%2Fimages%2Ffb_placeholder.png%3F1527840666&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/45dea947c3f72feb0a58289bdc623368/href">https://medium.com/media/45dea947c3f72feb0a58289bdc623368/href</a></iframe><blockquote><strong>Origin</strong>: Is strengthened by exercise, and must even slightly modify the food, the habits, and the whole economy of the race.</blockquote><ul><li><strong>DS</strong>: Is <strong>strentened </strong>by exercise and must even slightly <strong>moify </strong>the food the habits and the whole economy of the race.</li><li><strong>Google</strong>: Is strengthened by exercise, and must even slightly modify the food, the habits, and the whole economy of the race.</li><li><strong>MS</strong>: Is strengthened by exercise, and must even slightly <strong>modified </strong>food, <strong>habets, in </strong>the whole economy of the race.</li><li><strong>IBM</strong>: Is strengthened by exercise, and must even slightly<strong>.</strong> Modify the food, the habits, <strong>in </strong>the whole economy of the <strong>grace</strong>.</li><li><strong>Temi</strong>: Is strengthened by exercise, and must even slightly modify the food the habits and the whole economy of the race.</li><li><strong>nVoq: </strong>is strengthened by exercise, and must even slightly modify the food, <strong><em>[?]</em></strong> habits, and the whole economy of the race.</li><li><strong>Wit.AI</strong>: is strengthened by exercise and must even slightly <strong>modified </strong>food habits and the whole economy of the race</li><li><strong>Kaldi (tedlium)</strong>:<strong><em>his trenton </em></strong>by exercise karma and must even slightly <strong><em>modified </em></strong>the food habits and the whole economy <strong><em>over </em></strong>the <strong><em>grace</em></strong>.</li><li><strong>Dragon Pro 15</strong>: Is strengthened by exercise, and must even slightly modify the food, <strong><em>[?]</em></strong> habits, and the whole economy of the race.</li></ul><h4>Alice quote II (harder to recognize)</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttp%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F453101805%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Famit-be%2Falice2&amp;image=http%3A%2F%2Fa1.sndcdn.com%2Fimages%2Ffb_placeholder.png%3F1527840666&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/f9d0b866ffdce47a331ba04cef8a9dea/href">https://medium.com/media/f9d0b866ffdce47a331ba04cef8a9dea/href</a></iframe><blockquote><strong>Origin</strong>: Outside, Alice hears the voices of animals that have gathered to gawk at her giant arms. The crowd hurls pebbles at her</blockquote><ul><li><strong>DS</strong>: <strong><em>ooutside an s sheres</em> </strong>the voice of <strong><em>animal </em></strong>that have gathered to <strong><em>godbiter ghiant </em></strong>arms the crowd <strong><em>hurl’s devils</em> </strong>at he</li><li><strong>Google</strong>: Outside Alice, <strong><em>here’s </em></strong>the voice of animals that have gathered to <strong><em>God Couture</em></strong> giant arms the crowd <strong><em>hurdles Devil’s</em></strong> at her.”</li><li><strong>MS</strong>: Outside Alice <strong>here’s </strong>the voice of animals that have gathered to <strong><em>goget are</em> </strong>giant arms the crowd <strong><em>herels bevels</em> </strong>at her.</li><li><strong>IBM</strong>: Outside Alice hears the voice of animals that have gathered to gawk at her giant arms. The crowd hurls pebbles at her.</li><li><strong>Temi: </strong>Outside <strong><em>Ellis </em></strong>hears the voice of animals that have gathered to <strong><em>[?]</em></strong> giant arms. The crowd hurls pebbles at her.</li><li><strong>nVoq</strong>: outside <strong><em>else</em></strong><em> </em>hears the voice of <strong><em>animal </em></strong>that<em> </em><strong><em>I have</em></strong> gathered to <strong><em>guard a</em> </strong>giant arms the <strong><em>wound heals bevels</em></strong> at her</li><li><strong>Wit.AI</strong>: <strong><em>okay and this here’s</em></strong> the voice of animals that have gathered <strong><em>to got to </em></strong>trying to <strong><em>harm the crown </em></strong>hurls <strong><em>levels of </em></strong>her</li><li><strong>Kaldi (tedlium)</strong>: outside <strong><em>else here is</em></strong> the voice of <strong><em>animal health care</em></strong> <strong><em>other two battered</em></strong> <strong><em>trying to</em></strong> arms the <strong><em>crown </em></strong>hurls <strong><em>levels </em></strong>at her.</li><li><strong>Dragon Pro 15</strong>: Outside, Alice hears the voice of animals that have gathered to <strong>God with </strong>her giant arms. The crowd hurls <strong>levels that are</strong></li></ul><p>Summing it all up, I opted to count <strong>“stumbles”</strong>, which means counting whole blocks of errors as a single one, I chose this methods since advanced language models tend to over-correct for mistakes that skew grammar or context, thus dragging the error across a few words.</p><h4><strong>Totals</strong>:</h4><ul><li><strong>IBM</strong>: 2 stumbles</li><li><strong>Temi</strong>: 2 stumbles</li><li><strong>Dragon</strong>: 3 stumbles</li><li><strong>Google</strong>: 3 stumbles</li><li><strong>MS</strong>: 5 stumbles</li><li><strong>Wit.AI</strong>: 6 stumbles</li><li><strong>D.Speech</strong>: 8 stumbles</li><li><strong>nVoq</strong>: 9 stumbles</li><li><strong>Kaldi</strong>: 12 stumbles</li></ul><p>In all fairness, I have to admit that choosing the last quote as difficult, was due to Google having trouble with it when I was playing around, so you may give Google half a point off for that :). The distance between IBM and Google is not really enough for me to Judge, and it’s obvious that we should have a real showdown with some long form speech coming up below.</p><p>In general, we can see the state of the open source vs. the cutting edge, we can also see that no engine is perfect, they still have a way to travel to reach human level understanding, despite recent accuracy claims.</p><p>nVoq are specialists in healtcare text and from what I saw they do shine in that field, so I decided that my final showdown will include two final test samples: <strong>long form</strong>, and <strong>medical speech</strong>.</p><h4><strong>Interim conclusions:</strong></h4><p><strong>IBM,Google &amp; Dragon pass to the next stage</strong>. <br><strong>nVoq will also be in the finals </strong>as it is a representative of the medical dictation specialization, so it isn’t really in the same category and not trained on similar voice content.</p><p><strong>Finalists: IBM, Google (</strong>general speech<strong>), nVoq (</strong>medical speech<strong>)</strong></p><h4><strong>Bonus test: WaveNet-D voice sample</strong></h4><p>Since my voice is not perfectly canon as I may have a slight Hebrew accent, I wanted to have a perfect English sample of the last sentence which heavily confused google &amp; MS. To achieve this, I used the google TTS engine to produce a state-of-the-art WaveNet speech sample from text. After some base64 decoding &amp; conversion to 16Khz I ran the same upload routine with all the previously tested services…</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fw.soundcloud.com%2Fplayer%2F%3Furl%3Dhttp%253A%252F%252Fapi.soundcloud.com%252Ftracks%252F453101796%26show_artwork%3Dtrue&amp;url=https%3A%2F%2Fsoundcloud.com%2Famit-be%2Falice2-wavenet&amp;image=http%3A%2F%2Fa1.sndcdn.com%2Fimages%2Ffb_placeholder.png%3F1527840666&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=soundcloud" width="800" height="166" frameborder="0" scrolling="no"><a href="https://medium.com/media/72e9a2199583dc232330c50016f86742/href">https://medium.com/media/72e9a2199583dc232330c50016f86742/href</a></iframe><blockquote><strong>Origin</strong>:<strong> </strong>Outside, Alice hears the voices of animals that have gathered to gawk at her giant arms. The crowd hurls pebbles at her.</blockquote><ul><li><strong>DS</strong>: <strong><em>utside </em></strong>alice <strong><em>heres </em></strong>the voices of animals that have gathered <strong><em>do goc et</em> </strong>her giant arms the crowd hurls pebbles at her</li><li><strong>Google</strong>: Outside Alice hears the voices of animals that have gathered to <strong><em>God that are </em></strong>giant arms the crowd <strong><em>rolls </em></strong>pebbles at her.</li><li><strong>MS</strong>: Outside Alice <strong><em>here’s </em></strong>the voices of animals that have gathered to <strong><em>got </em></strong>that are giant arms the crowd <strong><em>girls </em></strong>pebbles at her.</li><li><strong>IBM</strong>: Outside Alice hears the voices of animals that have gathered to gawk at her giant arms. The crowd hurls pebbles at her.</li><li><strong>Temi</strong>: Outside Alice<strong>,</strong> <strong><em>here’s </em></strong>the voices of animals that have gathered to Gawk at her giant arms. The crowd hurls pebbles at her.</li><li><strong>nVoq</strong>: outside Alice <strong><em>years </em></strong>the voices of animals that have gathered to <strong><em>gawker</em></strong><em> </em>giant arms the crowd <strong><em>rose</em></strong><em> </em>pebbles at her</li><li><strong>Wit.AI</strong>: outside <strong><em>dallas here’s</em></strong> the voices of animals that have gathered to gawk at <strong><em>your </em></strong>giant arms the crowd <strong><em>rules </em></strong>pebbles at her</li><li><strong>Kaldi (tedlium): </strong>outside alice <strong><em>here’s </em></strong>the voices of animals that have gathered to gawk at <strong><em>a</em></strong> giant arms the crowd <strong><em>rose petals</em></strong> at her.</li><li><strong>Dragon Pro 15</strong>: Outside, Alice hears the voices of animals that have gathered to gawk at her giant arms. The crowd <strong>rose </strong>pebbles at her</li></ul><p>So we got the pebbles part out of the way, but it seems that hurling just ain’t something that the tech giants do — they prefer to <strong>roll</strong> <strong>or girl </strong>them instead (I wonder what would girling a pebble mean). I get not understanding “Gawk” since it is a rather seldomly occurring word.</p><h3><strong>Stage II: final showdown</strong></h3><p>For the final stage I wanted to check 2 different fields: Best all around speech recognition, and best medical field speech transcription. I chose to do this since as far as I know, computerized dictation is currently mainly used for story telling (writers), and for medical diagnosis reporting (doctors). All the voice in this test is recorded by me and can be found in the <a href="https://github.com/amitbet/speech/">github repo</a> which accompanies this article. Punctuation is handled by all engines, so no freebies this time around…</p><h4><strong>Medical transcription </strong>test</h4><blockquote><strong>Origin: A meld of two Diagnoses from online sites<br></strong>69-year-old female with a history of smoking, asthma and bronchitis now with productive cough intermittently for several months.Left medial foot and ankle pain and swelling. Plantar metatarsal pain for 5 weeks. No known trauma. Possible free air under the diaphragm. On a chest x-ray for what appeared to be shortness of breath she was found to have what was thought to be free air under the right diaphragm. No intra-abdominal pathology.</blockquote><p><strong>nVoq</strong>:<br>69-year-old female with a history of smoking, asthma and bronchitis now with productive cough intermittently for several months. <br>Left medial foot and ankle pain and swelling. Plantar metatarsal pain for 5 weeks. No known trauma. Possible free air under the diaphragm. <br>On a chest x-ray for what appeared to be shortness of breath she was found to have what was thought to be free air under the right diaphragm. No intra-abdominal pathology.</p><p><strong>Google</strong>:<br>69 year old female with a history of smoking, asthma and bronchitis now with productive cough intermittently for several months. <br>Left medial foot and ankle pain and swelling. Plantar metatarsal pain for 5 week <strong><em>There’s </em></strong>No known trauma. Possible <strong><em>for year</em></strong> under the diaphragm. <br>On the chest x-ray for what <strong><em>[?] the</em></strong> shortness of breath she was found to have<strong><em>.</em></strong> What was thought to be free <strong><em>are </em></strong>under the right diaphragm. No intra-abdominal pathology.</p><p><strong>IBM</strong>:<br>69 year old female with a history of smoking, <strong><em>Asman </em></strong>bronchitis now with productive cough intermittently for several months. <strong><em>Lift </em></strong>medial foot and ankle pain and swelling. <strong><em>Lanzar </em></strong>metatarsal pain for 5 weeks no known trauma. Possible free air under the diaphragm. On the chest X. ray for what <strong><em>the peer</em></strong> to be shortness of breath she was found to have what was thought to be free air under the right diaphragm <strong><em>period </em></strong>no <strong><em>intractable </em></strong>pathology.</p><p><strong>MS</strong>: 69 year old female with a history of smoking, <strong><em>after this</em></strong> now with productive cough intermittently for several months <strong><em>period </em></strong>left medial foot and ankle pain and swelling. <strong><em>plantare</em></strong> metatarsal pain for 5 weeks. <strong><em>known from a period </em></strong>possible <strong><em>freear</em></strong> under the <strong><em>front</em></strong>. on the chest X Ray for what appeared to be shortness of breath she was found to have what was not to be free air under the right diaphragm. no <strong><em>intradermal patala G.</em></strong></p><p><strong>Dragon Pro 15</strong>: 69-year-old female with a history of smoking, <strong><em>asked my</em></strong> bronchitis. Now, with productive cough intermittently for several months. Left medial foot and ankle pain and swelling. Plantar metatarsal pain for five weeks. No known trauma. Possible free air under the diaphragm. On <strong><em>the </em></strong>chest x-ray for what <strong><em>the peer</em></strong> to be shortness of breath. She was found to have what was thought to be free air under the right diaphragm. No intra-abdominal pathology.</p><h4>Story telling test</h4><blockquote><strong>Origin: Futurama, Episode 1 “space pilot 3000”</strong><br>Look Leela, I don’t understand this world but you obviously do, so I give up. <br>If you really think I should be a delivery boy, I’ll do it. He holds out his hand to Leela. She gets the implant gun ready. Fry cringes and looks away. The gun clicks but Fry feels nothing. He opens his eyes and sees Leela drop her own chip on the floor. Your chip, What are you doing?</blockquote><p><strong>Google</strong>:<br>Look Lila, I don’t understand this world, but you obviously do, so I give up. If you really think I should be a delivery boy, I’ll do it. He holds out his hand to Lila, She gets the implant<strong><em>s</em></strong> gun ready. Fry cringes and looks away. The gun clicks, but fry feels nothing. He opens his eyes and sees Lila drop her own chip on the floor. <strong><em>You’re </em></strong>chip, What are you doing?</p><p><strong>MS</strong>:<br>Look lela, I don’t understand this world but you obviously do, so I give up. if you really think I should be a delivery boy, I’ll do it. he holds out his hand to lila, she gets the implant gun ready. <strong><em>Frank whinges</em></strong> and looks away. the gun clips but <strong><em>Friday </em></strong>feels nothing. he opens his eyes and sees <strong><em>lelord </em></strong>drop her own chip on the floor. your <strong><em>trip</em></strong>, what are you doing?</p><p><strong>IBM</strong>:<br>Look Leila, I don’t understand this world but you obviously do, so I give up <strong><em>period </em></strong>if you really think I should be a delivery boy, I’ll do it. He holds out his hand to Leila <strong><em>come </em></strong>she gets the implant gun ready. <strong><em>Franklin gyms</em></strong> and looks away. The gun clicks but <strong><em>Frank </em></strong>feels nothing. He opens his eyes and sees Leila drop her own chip on the floor. Your chip. <strong><em>Comma </em></strong>what are you doing?.</p><p><strong>nVoq</strong>:<br>Look leila <strong><em>come </em></strong>I do not understand this world but you obviously <strong><em>to</em></strong>, so I give up. If you really think I should be a delivery boy, <strong><em>I will</em></strong> do it. He holds out his hand to Lila <strong><em>come </em></strong>she gets the implant <strong><em>can </em></strong>ready. Frei <strong><em>quenches </em></strong>and looks away. The gun clicks but Fry feels nothing. He opens his eyes and <strong><em>C’s</em></strong> leila drop her own chip on the floor. Your chip, what are you doing?</p><p><strong>Dragon Pro 15</strong>: Look Leela, I don’t understand this world, but you obviously do, so I give up. If you really think I should be a delivery boy, I’ll do it. He holds out his hand to Leela, she gets the implant gun ready. <strong><em>Fred </em></strong>cringes and looks away. The gun clicks but Fry feels nothing. He opens his eyes and sees Leela drop her own <strong><em>shape </em></strong>on the floor. Your chip, what are you doing?</p><h3>Conclusions:</h3><p>AI and Speech recognition have gone a long way in the last 5 years or so, computerized dictation and translation used to be so daft, they made me laugh so much my stomach hurt the first few times I tested them. Now it is almost good, churning out flawless sentences 3 times out of 4 with a good engine. I might even consider dictating my next medium article to Google docs, or MS dictate. there is also a fair amount of effort going into open source voice recognition, which is not trivial given the level of investment that designing such an algorithm entails. Just footing the bill for the model training hardware is impressive, not to mention the specialized man power.</p><p><strong>Best all-around speech recognition: [Google / Nuance]<br></strong>When dictating whole paragraphs to the engine you can see where Google really shines, it has excellent context understanding and has no problem when parsing punctuation. Despite IBM’s previous precision in the first stage of the tests, it is not perform as well as google’s engine when dictating long pieces of text. Dragon has one more error than Google on the story text, but less on the medical and my guess is that when using the medical engine it will be much better on that, it is still not enough in order to pass judgement but I would still stick with google as it is cheap as an engine &amp; free when using Google Docs, also it’s much easier to obtain.</p><p>I used Nuance Dragon 15 Pro, Which is not a medical voice engine, it is indeed a very good speech engine, and lives up to it’s name as a market leader. It did have the advantage of doing a short 3 sentence voice setup to tailor the engine to my voice, but it was just a short paragraph to read, so I won’t dock it any points for that.</p><p><strong>Medical speech: [nVoq]<br></strong>The differentiation from Google is even more apparent when freely playing with the engines, and when considering the fact that physicians have a close to zero tolerance for transcription errors.</p><p>It seems that the last 10% of recognition is hard for all engines alike and that a human brain has much more than a mere neural net with a language model behind it. So don’t expect perfect results, but if you are willing to correct a few errors, dictation can be a good option for you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f7d09c346d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>