<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Srdjan Delić on Medium]]></title>
        <description><![CDATA[Stories by Srdjan Delić on Medium]]></description>
        <link>https://medium.com/@sdremthix?source=rss-741be2c6b322------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*EKs2AFZg9LQLUe8BX7K0VA.jpeg</url>
            <title>Stories by Srdjan Delić on Medium</title>
            <link>https://medium.com/@sdremthix?source=rss-741be2c6b322------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:58:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sdremthix/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Is The Huawei Mate 20 Pro the ‘Mate’ for Time]]></title>
            <link>https://sdremthix.medium.com/is-the-huawei-mate-20-pro-the-mate-for-time-e1484c0cd609?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/e1484c0cd609</guid>
            <category><![CDATA[product-reviews]]></category>
            <category><![CDATA[smarthphone]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[devices]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Fri, 13 Dec 2024 22:16:59 GMT</pubDate>
            <atom:updated>2024-12-14T22:12:04.161Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1JnAdgcSwXYc6lc5JXBbiA.png" /><figcaption>Huawei Mate 20 Pr</figcaption></figure><p>We are nearing the end of 2024 and the bombardment of nostalgic, holiday-themed Instagram and TikTok videos floods my smartphone screen, not that I mind <em>doomscrolling</em> until the early morning hours, that is. Probably due to some specific Meta algorithm or witchcraft (my money is on witchcraft) I keeping getting these old Nokia, Samsung, early iPhone days, video compilations of some of my favorite devices ever. Legendary and imaginative designs procession-tailored to almost any individual-liking. You want a specific form factor, the manufacturers got you covered. Want a <a href="https://en.wikipedia.org/wiki/N-Gage">taco-shaped</a> handheld game device that occasionally moonlights as a smartphone? Gotcha covered! Want a small clamshell phone that can stream TV signals on a mighty 65536-color TFT display? <a href="https://www.gsmarena.com/nec_e616-533.php">NEC </a>got your back… If you were from South Korea in the early 00’s that is…</p><p>All this nostalgia scrolling on my 2K OLED Huawei <a href="https://www.gsmarena.com/huawei_mate_20_pro-9343.php">Mate 20 Pro</a> curved display (smooth transition, checked), got me thinking of my current main smartphone driver. My ‘Mate’. My ‘Pro’. I had this device since the late 2018, and to the moment of writing this post it hadn’t skipped a beat. I had a lot of favorite and memorable daily phone/smartphone drivers, starting from the legendary <a href="https://www.gsmarena.com/nokia_6150-10.php">NOKIA 6150</a> (still working with 60% of battery left) up to the <a href="https://www.gsmarena.com/samsung_galaxy_s9-8966.php">Samsung S9</a> before buying this Huawei Mate. All served their purpose, and were faithful until the day they weren’t. Most of them being steamrolled by technology, innovation, and time. The average lifespan of my main drivers was around 2 years as the principal devices of the household. The ‘Mate’ was/is the only device that keeps pushing the limits. The primary reason I’ve bought this device was it’s then-incredible camera system. On average, it was one of the highest scores on <a href="https://www.dxomark.com/huawei-mate-20-pro-camera-review-triple-cam-juggernaut/">DXOMARK</a> at the time. As I was dabbing into DSLR photography myself, that was one of my primarily requirements besides the flagship SOC specifications (future-proof and good for Android development). I can’t count the times I was asked about what device I was using and how the night/day photos turned out to be so good. To this day even. <br>With each year and new Samsung or Apple flagship, I’ve contemplated switching to the most promoted one, and then realized that the minor incremental improvements smartphone manufactures make each year are truly <em>minimal</em>, and usually redundant. So many devices were introduced with my <em>digits</em>, and while one device maybe had the better battery life, it balanced it out maybe with a worse camera or display, and that was the case for quite some time. <em>EONS </em>in the terms of technology improvements.</p><p>OK, “your requirements changed over the years” — you might say at this point. Yes, but very minimal. The smartphone, to this day, still completes 90% of my requirements. Is it responsive? Yes, it is. To be precise, this is the only device I had for more than 3 months, that never showed it’s age. I can still play new games on it. The battery (4200 mAh Li-Po) lasts at least a day at an average of around 5 hours of screen-on time. The display is fantastic and the high pixel density shows its virtues. The size and weight is perfect for my hands. Toughness,call quality, software? Check, check, and check. Huawei long supported this device with constants software updates and improvements. You see the point I’m trying to make?</p><p>Speaking of the software, I could be wrong, but this might be the last or at least one of the last Huawei flagship devices that came with Google Play Services and Google application (GMail, Maps, Chrome, etc) support out-of-the box, probably influencing its aftermarket price. This is a story in all by itself, so I’ll just move pass it at this time.</p><p>Drawbacks? Of course there are some. With an internal storage capacity for 128GB it can be a bit crowded in the space department, especially in 2024 where filesizes of the most basic of cat GIF’s is a gorillion kilobytes of data. After all these years, a lot of memories are accumulated and the need for increased storage slowly creeps in. There’s a memory card slot, so that potential ‘issue’ is addressed there. Support for newer Android versions (Android 12 and up), has halted, and it would be great to have some of the features new Android versions bring with them.</p><p>Will I replace my “Mate’? Most definitely, yes. I’m in the tech world, and sooner than later a device will fall into my hands that’ll check all the boxes the Mate 20 Pro did that late December evening of 2018 when I bought it. Until that day arrives, I’m glad that I had the chance to be surprised in a positive way with a <em>faithful</em> <em>companion</em> such as the Huawei Mate 20 Pro.</p><p>I’m interested to hear what device, lets be materialistic here for a bit, has held your imagination or was your go-to device despite its age? No matter the technology. It can be a car, bicycle, smartphone, refrigerator, anything. Comment below.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e1484c0cd609" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Android Fastlane Tutorial]]></title>
            <link>https://sdremthix.medium.com/android-fastlane-tutorial-8819d3d2d24c?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/8819d3d2d24c</guid>
            <category><![CDATA[tutorial]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[fastlane]]></category>
            <category><![CDATA[deployment]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Fri, 01 Mar 2024 18:03:08 GMT</pubDate>
            <atom:updated>2024-03-01T19:49:54.568Z</atom:updated>
            <content:encoded><![CDATA[<p>Automating an Application Deployment Process</p><figure><img alt="Srdjan Delić — SDRemthix Android tutorial header image of the official Android logo" src="https://cdn-images-1.medium.com/max/1024/1*kC_-Q4g5W0c1vY2kLZVp9w.png" /></figure><p>Let’s not waste any time, if you’re an <a href="https://play.google.com/console/signup">Android developer</a> with an <a href="https://www.apple.com/mac/">Apple Mac</a> development machine, and want to automate the build, testing and deployment process using the Fastlane framework, you’re at the right place.</p><p>This tutorial as an addendum to the existing setup process available at the official Fastlane documentation <a href="https://docs.fastlane.tools/">page</a>. If you’re experiencing issues with your Fastlane setup, you can use this article as a checklist of sorts or go directly to the troubleshooting section of this article for the more common issues when running Fastlane deploy.</p><h3>Fastlane Setup Prerequisites</h3><p>First things first, visit the following page and make sure that you’ve obtained and secured the needed <strong>*.json</strong> API file that is tied to your existing Android Developer account:</p><p><a href="https://docs.fastlane.tools/getting-started/android/setup/">https://docs.fastlane.tools/getting-started/android/setup/</a></p><ul><li>You have already created the project for your app on the <a href="https://play.google.com/console/about/">Google Play Console</a></li><li>You have generated the <em>release </em>signing credentials for your app (keystore <strong>*.jks </strong>file and password, alias, alias password)</li><li>On a Mac machine, make sure you are in the right shell and that JAVA_HOME is defined (best use the same one as in your<a href="https://developer.android.com/studio"> Android Studio Environment</a>)</li></ul><h3>Android Project Setup</h3><ol><li>Create a new folder in the root directory of your app and call it <strong>‘secure’</strong></li><li>Add the secure folder to .<strong>gitignore</strong> rules: <strong>/secure/*</strong></li><li>Copy your already created <strong>*.jks </strong>and Google Developer Cloud API <strong>*.json </strong>(created in the Fastlane setup section ) files into the newly created <strong>secure </strong>root app directory</li><li>In your <strong>local.properties </strong>add the Android app-signing information in the key-value format (make sure that the information you enter here is valid):</li></ol><p><strong>KEYSTORE_PATH</strong>=<em>PATH_TO_KEYSTORE_ON_LOCAL_MACHINE</em></p><p><strong>KEY_ALIAS</strong>=<em>APP_ALIAS</em></p><p><strong>KEY_PASSWORD</strong>=<em>APP_ALIAS_KEY_PASSWORD</em></p><p><strong>STORE_PASSWORD</strong><em>=APP_STORE_PASSWORD</em></p><p>We will create a new <em>release </em>signing configuration. In your <strong>app </strong>module’s <strong>build.gradle</strong> file, inside the <strong>android </strong>closure, add the following code at the beginning:</p><pre><br>signingConfigs {<br>        release {<br>            def keystoreProperties = new Properties()<br>            def keystorePropertiesFile = rootProject.file(&#39;local.properties&#39;)<br>            if (keystorePropertiesFile.exists()) {<br>                keystoreProperties.load(new FileInputStream(keystorePropertiesFile))<br>            }<br>            storeFile file(keystoreProperties[&#39;KEYSTORE_PATH&#39;])<br>            keyAlias keystoreProperties[&#39;KEY_ALIAS&#39;]<br>            keyPassword keystoreProperties[&#39;KEY_PASSWORD&#39;]<br>            storePassword keystoreProperties[&#39;STORE_PASSWORD&#39;]<br>        }<br>    }<br><br></pre><p>6. Edit the <em>buildTypes </em>closure inside the same <em>android </em>closure block and add the following line inside the <strong>release </strong>block:</p><pre>signingConfig signingConfigs.release</pre><p>7. Initialize Fastlane for the project and follow the setup console prompts to complete the Fastlane project setup. Run the following command inside the root directory of your project:</p><p><strong>fastlane init</strong></p><p>8. In the newly created <em>fastlane </em>directory we first need to edit <strong>AppFile </strong>and provide/validate the information for the <strong>json </strong>API key file path and app <strong>package name</strong>:</p><pre>json_key_file(&quot;./secure/DEV_API.json&quot;)<br>package_name(&quot;your.app.package.name&quot;)</pre><p>9. Next we define the steps for Fastlane to take when we run the script. For the sake of brevity, I’m going to only showcase the final deployment step for the <strong>bundle release</strong> deployment:</p><pre><br>platform :android do<br><br> desc &quot;Deploy a new version to the Google Play&quot;<br> lane :deploy do<br>   gradle(<br>     task: &#39;clean bundle&#39;,<br>     build_type: &#39;Release&#39;<br>   )<br>   upload_to_play_store # Uploads the AAB built in the gradle step above and releases it to all production users<br> end<br><br>end</pre><p>10. Check if Java runtime is exported in the JAVA_HOME environment value (see below for troubleshooting options). You can run the following command in the terminal to validate if Java is installed (double dash):</p><p><strong>java -version</strong></p><p>11. Continue with executing the Fastlane deployment. When running the first time, it can take some time until all of the dependencies and metadata information are downloaded and the build and deployment steps commence. This step largely depends on your build machine performance and the speed of your local internet connection</p><h3>Run Fastlane Deploy</h3><p>After the steps above have been completed, you can finally run Fastlane deploy with the following command:</p><p><strong>bundle exec fastlane deploy</strong></p><p>Based on the <a href="https://bundler.io/">Bundler</a> configuration on you machine, you might need <strong>sudo </strong>privileges. Run this instead:</p><p><strong>sudo bundle exec fastlane deploy</strong></p><h3>Troubleshooting</h3><h4>Java Runtime Not Found</h4><p>In case you’re getting java runtime not found, try these steps in the terminal:</p><p><strong>/usr/libexec/java_home -V | grep jdk</strong></p><p>You should get something like this in the output:</p><pre>Matching Java Virtual Machines (1):<br>1.8.301.09 (x86_64) &quot;Oracle Corporation&quot; - &quot;Java&quot; /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Home</pre><p>Let’s add or update the JAVA_HOME path using <a href="https://www.vim.org/">vim</a> :</p><p><strong>vim .zshrc</strong></p><p>Proceeding with adding the following information (make sure that the correct Java <a href="https://www.oracle.com/java/technologies/downloads/">JDK</a> path is entered, below is just an example):</p><pre>export JAVA_HOME=/Users/your_device_account/Library/Java/JavaVirtualMachines/java_version/Contents/Home<br>export PATH=$JAVA_HOME/bin:$PATH</pre><p>Make sure you replace the JAVA_HOME path with the one identified with which java</p><p>Depending on the profile you are using for the shell, it should work for zsh too.</p><p>To apply the new changes run (or <strong>.barshrc</strong>, or <strong>.zsh</strong>, etc.):</p><p><strong>source .zshrc</strong></p><p>This would probably not persists and when you restart the terminal, you’ll face the same problems.</p><h4>Gradlew Permission Denied</h4><p>In case if you’re getting permission denied, run the following command to give write permission to gradlew:</p><p><strong>chmod +x gradlew</strong></p><h4>Gradle Deprecated or Unrecognized VM Options</h4><p>In you are experiencing build issues with Gradle, make sure that you are not using deprecated gradle jvmargs or similar:</p><p><strong>Unrecognized VM option ‘MaxPermSize=512m’</strong></p><p>Remove or replace the arguments with ones that are supported by the new versions of the build system.</p><h3>Final Thoughts</h3><p>The steps provided in this article are not perfect and are simply my own way of getting Fastlane quickly up and running for new and existing projects. I’m sure that most of things described and showcased can be further improved upon and streamlined for an overall better developer experience.</p><p>If you found this tutorial useful, feel free to leave a comment and feedback.</p><p>Thank you for reading, and happy coding!</p><h3>References</h3><p>Fastlane official documentation: <a href="https://docs.fastlane.tools/">https://docs.fastlane.tools/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8819d3d2d24c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Discovering: MLOps]]></title>
            <link>https://sdremthix.medium.com/discovering-mlops-a49ef3139696?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/a49ef3139696</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Sun, 04 Jun 2023 23:16:10 GMT</pubDate>
            <atom:updated>2023-06-04T23:16:10.713Z</atom:updated>
            <content:encoded><![CDATA[<h4>Episode 1: An symbiotic dance of two highly valued fields in contemporary data-shifting environments</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a-BUPZQ07K9QxMh9I7oydA.png" /></figure><p>In two of my previous AI/ML — oriented <a href="https://sdremthix.medium.com/what-is-aiaas-c33b799b2a6c">articles</a>, I’ve very briefly touched on the subject of MLOps. In this article, one that could be considered a follow-up on the previous ones, I’ll try to introduce this relatively new concept that could help anyone interested in the topic transition from the existing machine learning practices into enterprise level standards.</p><p>Right from the start I should come clean and say that I was mistaken in my initial thoughts when hearing about machine learning operations (MLOps). I though it was a merging of ML and traditional DevOps. While that might be the general case and one would be correct for making such a statement, the real ‘thing’ is a bit more complex than that and it involves aspects that are much more closer to idiomatic machine learning workflows than traditional software development principles. So, what is MLOps?</p><p>MLOps represents an abstract idea of an end-to-end looping cycle of data science, machine learning and developer operations (DevOps) practices and principles for an optimized system development cycle. By utilizing an automated process that combines components ranging from data extraction and preparation, through model training and, finally, deployment and monitoring, one can significantly reduce the model development cycle and solidify high quality standards.</p><p>In this <em>discovering </em>series of articles, we’ll cover the basic definitions of the key concepts around MLOps, the benefits, demystifying its principles, and some of the challenges that have emerged in the past few years. Along the way, there will be a <em>sprinkle </em>of an example project. So, let’s move on with introducing some key concepts.</p><h3>What is DevOps?</h3><p>DevOps stands for “development operations” and is meant to represent a set principles, tools, culture and methodologies that have an end goal do bridge gaps in traditional teams such as QA, development and IT infrastructure, when producing new and maintaining existing solutions. It is meant to speed up and maintain a certain quality threshold in the product development and deployment lifecycle. Some of the typical steps in a DevOps workflow consist everything from task planning, designing, code commits, linting and testing the said code and finally, deployment, delivery and monitoring for the whole solution. DevOps introduces the concept of continuous deployment and integration, effectively gatekeeping any system anomalies to slip through different stages of the product lifecycle. This can manifest in better overall product quality and general service availability since we are in no need to compromise existing and running production services with potential problematic states. More and more tools and concepts are introduced that expand the reach of DevOps to cover as much as possible of the product workflow from the conception phase to the very well developed deployment and monitoring phase. Next, we’ll look at some key principles surrounding DevOps.</p><h4>Automation</h4><p>What’s there to say about this, perhaps, most important aspect of DevOps? Automation is the key that makes every other aspect of this process possible with enforcing effectiveness and repeatability of each task. Let’s be frank, as software developers, data scientists or machine learning engineers, we are prone to making mistakes when doing repeatable tasks. Automation can help us mitigate the risks taken when manually writing, triggering and executing task. We can simply write code, usually in the form of scripts, that can <em>react </em>to external factors that influence the whole product lifecycle.</p><h4>Architecture</h4><p>Choosing the right system architecture can be the deciding factor when cost reduction, scaling and maintenance are crucial driving forces for a solution. Closed-up architectures are by definition very isolated and dependant on API gateways in order to provide communication between various components, and, even more importantly, the various stages of the complete development to operations cycle. With good system architecture, we can predict what parts of the system are providing the most value to the overall product success criteria, and introduce entry points into which we can plug-into and interact with during each of the before mentioned stages in a typical DevOps workflow.</p><p>There’s no right and wrong answer when choosing an overall system architecture. There’s only architectures that fulfill the acceptance criteria. What proved to be successful are architectures that enable various implementation details for components to be abstracted from the rest of the system using a form of <em>contracts</em>. Contracts help the business logic for a solution to be independent of the concrete implementations for the said contracts. A huge benefit in this kind of an approach is that, at any stage of the DevOps cycle, we can replace anything, from libraries to the tooling used in the solution.</p><h4>Continuous Integration/Continuous Delivery (CI/CD)</h4><p>In the software development process, it is common to have some sort of centralized source code repository that serves as a single source of truth for the whole project. Usually, that repository is hosted on some cloud service like <a href="https://github.com/">GitHub </a>or <a href="https://about.gitlab.com/">GitLab</a>. When software developers merge their latest code changes, automated services can be configured that listen to those code changes and trigger automatic events such as build and testing processes. What this accomplishes, beside the time and error reduction of manually triggering those kind of events, is that issues can be caught very early on in the development cycle, much sooner than any production release schedule and also, if every defined criteria standards is successfully competed, maintain a constant release schedule.</p><h4>Inter-Team Communication</h4><p>Since DevOps throws such a wide net over the development cycle, different teams can be part of it. Inter-team communication is crucial for any product success. What DevOps helps with is that, no single team can work outside and in isolation on a product. A seamless transition is made between the end and beginning stages of each team’s work, with clear and understandable practical contracts between them that act like gateways for other teams. The end goal is to ensure minimal to none defect leaving the state one team on another that could lead to a waterfall effect, where one defect is propagated through all teams right until the final production deployment. There are also a set of sub practices focusing on clear inter-team communications such as ChatOps. The understanding of accountability and a good sense of team roles, lead to a more clearer vision of what the end product needs to be.</p><h4>Infrastructure as Code (IaC)</h4><p>IaC can be seen like a management layer for data provisioning centers where most of the configurations can be directly relayed to the underlying system resources by using machine readable configuration files. The goals is to abstract the hardware layer for the benefits of more consistent, testable, repeatable and automated deployment and management of the infrastructure layer.</p><h4>Monitoring and Logging</h4><p>At the end, we have to know how our system is performing. How resources are being utilized. The service health status with checks for warnings and errors. We can monitor how individual updates and changes affect the overall state of an solution and if there is need for any rollback to previous acceptable state. Logging and monitoring also helps in providing concrete data for potential stakeholders, enabling them to make key strategic decisions when moving forward with the project. Logging helps us identify issues early on, and shorten the team response times for addressing those issues, enabling the rest of the system to maintain its availability throughout this process.</p><p>With these principles in place, companies and individuals alike can create environments where fostering coding, building,testing and deployment stages can happen more frequently and, most importantly, more consistently.</p><h3>MLOps vs. DevOps</h3><p>With the DevOps part covered in the previous segment, we can proceed with differentiating MLOps to DevOps. It’s only natural to assume that MLOps is an extension of DevOps, they share 50% of their names, don’t they? We should thread carefully, because this kind of thoughts can be very misleading. Yes, MLOps shares the same culture and most of the practices that are introduced in DevOps, where they differentiate is where their end results live. While DevOps is mostly focused delivering an end product, MLOps focuses on machine learning models, their versioning and overall data management.</p><p>MLOps can definitely be seen as an extension to DevOps that specifically addresses issues and challenges with deployment and management of models within the AI/Machine Learning ecosystem for production environments. MLOps aims to cover the complete ML lifecycle, including research, data extraction and validation, model training, deployment, monitoring and inner loop (section lifecycle) refactoring with iterative improvements. With MLOps we are combining ML techniques with DevOps practices to achieve the necessary level of effectiveness, inter-team communication and consistency in the ML world that has already been proven in software development with DevOps.</p><p>A visual representation of the two can loosely be observed in the following diagrams:</p><figure><img alt="DevOps vs MLOps venn digramas" src="https://cdn-images-1.medium.com/max/1024/1*q7NdFwdwjJ9-CF__WdhAkA.jpeg" /></figure><p>Both, MLOps and DevOps put an emphasis on continuous improvements and continuous learning. One obvious differentiating point between DevOps and MLOps, is their respective <strong>domain </strong>knowledge areas. We’ve already mentioned that DevOps focuses on the software development lifecycle, but MLOps in addition to software development, requires knowledge of machine learning, data science and statistical analysis. Each of those additional fields being extremely vast in scope with engineers that are successfully joggling all of them being extremely valuable to industry giants.</p><p>There’s also a certain level of <strong>experimentation</strong> and <strong>iteration</strong> looping involved when trying to improve upon a certain model type in MLOps. The same can definitely be said about a software solution in DevOps, but the degree to which experimentation is involved in ML is more akin to performance optimization in software development to which the cost-to-effectiveness ratio drastically drops for each gained performance point. The complete opposite is true in ML. Fine-tuning a model can exponentially gain more and more usability and return on investment (ROI), with the norm usually being that at least one model iteration improvement being required in order to satisfy production criteria.</p><p>With the concept of MLOps being an extension to DevOps, versioning plays an even more important role. <strong>Data</strong> and <strong>model</strong> versioning are just as important as code versioning in the world of ML. At the end of a ML lifecycle, in the monitoring and logging stage, we have to consider all the factors that came into play when creating the output model. What was the input data, what was the data source, what parsing algorithm has been applied to it, to what extent has the data being cleaned-up before being released as a training model set? These all are just potential versioning opportunities regarding the model training data, but we also have the model versioning aspects to consider, such as model fine-tuning hyperparameters. It’s easy now to understand how much more intricate MLOps can be.</p><p>Needles to say, much of the tooling and infrastructure is considerably different than in standard DevOps. Anything from model serving, distributed training to monitoring web frameworks, all require to be bespoke-made with ML in mind. With good architecture oversight, we can mitigate the costs of different frameworks and tools with utilizing the efforts to make specialized API gateways with which we can plug-into our existing tooling.</p><p>The last important aspect to consider when comparing these two development culture practices is <strong>ethical</strong> considerations. Data privacy and model <em>fairness </em>are playing key roles in certain industries such as healthcare. This is specifically difficult topic to address and specialized teams are required for creating test cases and validations when targeting certain markets and User groups. Not every model and data source can be applied to niche domains where sensitive topics are present. Early efforts in publicly available AI/ML solutions, have produced less-than-ideal results when no ethical filters have been applied to them. We strive now to make solutions that can be adapted to more versatile scenarios.</p><h3>Why do We Need MLOps?</h3><p>The current state of things in the world of machine learning and data science is not suitable for rapid-track change requests and evershifting markets. The more complex models, by some thinking, more valued models, require an extensive amount of time to train, validate and ship to markets. Teams centered around AI/ML are thought to execute manual data experiments, data preparation, model training and providing final evaluation metrics. This can be extremely time consuming, labor intensive, inflexible, not reusable, and at the end, error prone. A prevailing thought, at the time of writing this article, is that machine learning needs to be around ten times faster from the initial steps of data gathering to market delivery! Sounds incredible, doesn’t it? MLOps has emerged with the sole purpose to bring order to this, very problematic, state of things when AI/ML is concerned.</p><h4>Data</h4><p>For an optimal MLOps experience, the gathering, analysis and clean up of the input data is fundamental. A quality result model is directly tied to the quality data upon which it is trained on. A complete sub section of MLOps is centered just around the data pipeline.</p><h4>Model and Data Drift</h4><p>What is usually omitted when talking about MLOps, are the concepts of data drift and model drift. Model drift refers to the handling of new requirements and change requests while data drift notifies the rest of the MLOps pipeline that the data we re interested in has changed. Both, model and data drift should have listening services attached to any change events and trigger automatic rebuilds or retraining of models.</p><h4>Project Structure</h4><p>While it might not seem as important as some other steps mentioned in this section, project structure can significantly boost the overall productivity of all teams involved. It makes the onboarding of new team members fully transparent and ensures that everyone is on the same page when resolving new issues and tackling new challenges. A good project structure as well as good documentation and planning, provide a common language that everyone involved can <em>speak.</em></p><h4>Experimentation</h4><p>Why <em>playing</em> around with your data can be extremely valuable. Software developers have debugging, machine learning engineers and data scientist have <em>experimentations.</em> A lot can be said from differentiating experimental cycles. Depending on the client requirements, the experimental phase can be significantly prolonged if no optimized steps are applied. ML engineers need to define specific criterias that can enable them to quickly iterate thought different data sets and input parameters in order to shorten the feedback loops and improve their workflow. The days of waiting days and months for a model to be trained should be reduced as much as possible. Not only will this help keep the costs down, but also change the overall perspective in how a ML workflow should look like.</p><p>Some aspects to consider when looking into reduction points when experimentation is considered:</p><ul><li><strong>Feature selection</strong> — What feature are we currently focusing on</li><li><strong>Algorithm selection </strong>— Switching between different algorithms should be as seamless as possible. This step requires quite a bit knowledge from the engineering team</li><li><strong>Hyperparameter tuning </strong>— We should be able to quickly pass in new parameters when constructing new models</li><li><strong>Model fitting </strong>— Model fitting refers to measurements of how well the training model adapts to new data that is similar to the data it was trained on. This is a fine measurements, since it’s generally regarded to walk a fine line between models that are too <strong>accurate </strong>and those that are not. This is crucial in defining relationships and patterns in the input, out and target data</li></ul><h4>Metadata Versioning</h4><p>Through this section we can observe that a lot of considerations can be made and there’s a lot of room where we can help differentiate different aspects of the MLOps lifecycle with versioning and labeling. Each of the <em>cogs</em> in the intricate engine that is MLOps can have it’s own distinct version that can potentially be reused in order to enhance attributes of the final deployment outcome. Each of those <em>cogs</em> should be made interchangeable with a component of the same type, but different in the metadata that define them. In order for the final monitoring, logging and analytical steps to be complete, we need to have as much insight in the complete roadmap, from the input data, its parameters through the model hyperparameters, in order to form calculated judgments when comparing the deployed models. Every single of the beforehand mentioned experimental runs, needs to be tracked of its metrics, this also applies to the source code versioning of the software development aspects of MLOps.</p><h4>Learning From Mistakes</h4><p>The complete culture and practices around MLOps have risen as answers to some previous mistakes made by ML engineers and data scientists. Some issues are hard to uncover in the traditional ML workflows. It’s easy to get carried away when working on a solution. May it be with new algorithms, tools or inner-team communication, we tend to get distracted with them and compromise our initial vision or workflow, resulting in some form of issues or errors. With a good foundation in MLOps cycles (logic-loops), we can pinpoint a lot of the problematic areas relatively quickly and address them in a timely manner without the need to stop the complete pipeline unless its some sort of critical issue that affects everything in our solution. We can also much more effectively define checkpoints in and around the ML lifecycle with custom linting, test, and static code analytical tools. Only when we identify the errors in our solutions, we can learn from them and further improve the very next iteration with fast market delivery.</p><h4>Automated Model Deployment and Validation</h4><p>Just the mere mention of the words ‘manual deployment’ can send shivers down the spines of software developers and that’s for a good reason. No matter how careful we are, the room for error increases with each of our manual tinkering with model deployment and validation. In order to increase efficiency and significant time savings, we need to introduce automation in our ML lifecycles. And that’s not even the end of it. With automation, we can talk about scaling our solutions based on the resources at hand. Standardization and consistency are also introduced in automated pipelines with version control, monitoring and quality assurance closely following all of those good practices that make difficult workloads more manageable. While I haven’t experimented with a lot of tooling for automated packaging and deployment when ML workflows are concerned, some tools were mentioned a lot in my research about this topic. Tools such as <a href="https://mlflow.org/">MLFlow</a>, <a href="https://airflow.apache.org/">Apache Airflow</a>, <a href="https://cloud.google.com/products/ai">Google Cloud AI platform</a>, <a href="https://azure.microsoft.com/en-au/products/machine-learning">Azure ML</a> and <a href="https://www.kubeflow.org/">Kubeflow</a> (<a href="https://kubernetes.io/">Kubernetes</a> extension), all provide good out-of-the box functions to be integrated with our solutions. While it might be tempting for an individual or company to create their own custom solution for automating the model deployment and validation, we should be careful, since the pitfalls can be significant if we do not fully grasp the scale and problems surrounding the ML lifecycle. It’s best to research each of those tools to see what best suits your own project budget and criteria.</p><h4>Ethical Checkpoints</h4><p>What happens when new models are trained on proprietary or licensed data? How many data scrapers are running through countless of data stored on web hosting platforms each day? I’ve briefly touched upon this subject matter in the previous section, but it might be good to emphasize the importance of this MLOps component since it’s something very specific to this domain.</p><p>User data privacy and security should be one of the top priorities if we deal with such data sources for our models. Propper anonymity, de-identification and even data encryption, should be made part of our MLOps workflows. Specialized auditing mechanisms can be implemented in ensuring that our security measures are up to date to the latest industry standards.</p><p>We should also mention the environmental impact that model training can have. Namely, depending on the model type and training data can exhort vast amounts of computing power and resources in general. A parallel can be made to what blockchain mining <a href="https://stekhn.github.io/gpu-price-surge-analysis/">impact</a> had on the graphics card (GPU) market and overall environment. At the time of this writing AI/ML are commanding significant resources, especially some of the more intensive <a href="https://en.wikipedia.org/wiki/Large_language_model">LLM’s</a>.</p><h4>Monitoring</h4><p>And how do we know that our models are working as expected? How they differentiate between incremental versions and even if they are available to our clients and customers? Of course, we can monitor them upon delivery. Monitoring and logging of interactions, input and response data, provide us with the necessary metric data that we can later analyze and work with in order to identify potential missteps, performance bottlenecks and other ways we could improve our models for the next step in our solutions roadmap. Monitoring metrics also enables us to showcase the results to any potential stakeholders and work on the feedback we receive.</p><h4>Workflow</h4><p>A typical MLOps workflow would look something like the following flowchart:</p><figure><img alt="MLOps workflow — SDRemthix" src="https://cdn-images-1.medium.com/max/1024/1*jkwO4qjwczMyeA-ffvVNyQ.jpeg" /><figcaption>MLOps workflow</figcaption></figure><h3>Benefits of MLOps</h3><p>We’ll keep this section short and to the point. This a sort-of summary of the previously mentioned attributes of MLOps, but it’s nice to hear them again in list form:</p><ul><li><strong>Scalability and Management</strong> — All parties involved adhere to the same culture and practices of creating more manageable and maintainable systems. Solutions made with the scalability and maintenance planned from the very beginning, have a better chance to deliver on product requirements. It can at first seem that progress is slow, especially in the very early stages of ML lifecycle, but its the foundation that we’ll rest the proceeding layers of our solution</li><li><strong>Reusability and Reproducibility</strong> — The simple things is that we need to reproduce outcomes of our model training depending on various scenarios. Reproducibility can help us better understand how the underlying data is being manipulated by the model and if it satisfies our progress criteria. As for re-usability, we can apply <a href="https://en.wikipedia.org/wiki/SOLID">SOLID</a> principles coupled with <a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a> programming, to ensure that components are as reusable and versatile as our business domain requires them to be. It also helps us to become better overall developers and produce high-value results</li><li><strong>CI/CD</strong> — We need to know to package, control and monitor our solutions and also where to identify logic gateways for each of the coupling components of our ML solution, so that the output is as production ready as possible. Automated steps will help each team to focus on what is of utmost importance in our solution and let the infrastructure tooling do the rest. When properly setup, of course</li><li><strong>Health and Governance</strong> — MLOps enables us to not only observe our models in action, but also to rapidly adjust them with various hyperparameters (depending on the model and algorithm) in order to fine-tune them in a effective way and ready them for production deployment</li><li><strong>Ethical AI Practices</strong> — From the start, MLOps introduces the ethical component in our workflows, ensuring that this highly sensitive industry is impacted by the real-world information, only when we need them. At the time of writing this article, we are witnessing more and more AI solutions that have the specific purpose in detecting other AI-generated content and report on any User rights violations or plagiarism for instance. Early identification of the possible sensitive areas in our training and input data, can elevate our ML solution and make it stand out in an emerging, competitive market</li></ul><h3>Final Note</h3><p>We’ll continue with this <em>discovering </em>series in the next article/episode, where we’ll look into an example MLOps project.</p><p>Are you already familiar with MLOps? What are your thoughts on the current state of machine learning delivery? As always, your feedback and thoughts are greatly valued, so feel free to leave a comment, question or message me.</p><p>Thank you for reading, and happy engineering!</p><h3>References</h3><ul><li>What is MLOps: <a href="https://www.youtube.com/watch?v=Io5t4ZqNDFQ&amp;ab_channel=KinetoKlub">https://www.youtube.com/watch?v=Io5t4ZqNDFQ&amp;ab_channel=KinetoKlub</a></li><li>What is MLOps? Why do you need it, and Where do you begin: <a href="https://www.youtube.com/watch?v=7izNbvw1t6Y&amp;ab_channel=VertaAI">https://www.youtube.com/watch?v=7izNbvw1t6Y&amp;ab_channel=VertaAI</a></li><li>MLops Explained: <a href="https://www.youtube.com/watch?v=ZVWg18AXXuE&amp;ab_channel=MicrosoftAzure">https://www.youtube.com/watch?v=ZVWg18AXXuE&amp;ab_channel=MicrosoftAzure</a></li><li>Introduction to MLOps: <a href="https://www.youtube.com/watch?v=Kvxaj6pHeVA&amp;ab_channel=PragmaticAILabs">https://www.youtube.com/watch?v=Kvxaj6pHeVA&amp;ab_channel=PragmaticAILabs</a></li><li>MLOps: <a href="https://ml-ops.org/">https://ml-ops.org/</a></li><li>MLOps Wiki: <a href="https://en.wikipedia.org/wiki/MLOps">https://en.wikipedia.org/wiki/MLOps</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a49ef3139696" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is AIaaS?]]></title>
            <link>https://sdremthix.medium.com/what-is-aiaas-c33b799b2a6c?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/c33b799b2a6c</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[ai-as-a-service]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 08 May 2023 04:07:50 GMT</pubDate>
            <atom:updated>2023-05-08T07:00:02.374Z</atom:updated>
            <content:encoded><![CDATA[<h4>All the power of AI on the Cloud</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uBcbig5L2fDZ-Z8QnWxtLw.png" /></figure><h4>Intro</h4><p>Is that some new technology or some terminology brewing on the horizon? It must be a new day indeed. OK, just recently I’ve found out about the concept around <a href="https://www.ibm.com/topics/artificial-intelligence">AI</a> as a Service (AIaaS), and wanted to explore this subject matter a bit more and share my thoughts and experiences with you. I should mention that I’m not an AI scientist or ML engineer (not yet, at least) before diving into any further content. I’m just a curious software developer with a healthy hunger for new technologies and their potential impact on our daily workflows and lives.</p><p>What I hope is that your curiosity is ignited by the end of this article and that you’ll continue with further exploring this new interesting world and emerging concepts around the fabulous topic of Artificial Intelligence. Let’s just jump in and see what is what.</p><p>Artificial Intelligence as a Service (AIaaS) is a new way in which companies and developers can access and utilize artificial intelligence capabilities to benefit their goals. In this article, we’ll dive into the concept of AIaaS, clarify common misconceptions, and explore a practical solution using some popular programming languages and frameworks.</p><h4>Beyond the Definition</h4><p>As mentioned earlier, AIaaS focuses around the delivery of AI power through cloud-based platforms and services. It enables organizations to access and utilize AI technologies without the need for in-house expertise, infrastructure, or significant investment. Which, all in by itself, can save countless of development and research hours. AIaaS includes a wide range of AI services, such as machine learning, computer vision (images), natural language processing (<a href="https://en.wikipedia.org/wiki/Natural_language_processing">NLP</a>), and even speech recognition, each offered by specialized cloud ad AI service providers.</p><p>What could be perceived as positive in having such powerful tools delivered and integrated in our own solutions (other than the already mentioned that is)? Some additional benefits can be one or more of the following:</p><ul><li>Scalability: AIaaS platforms are designed to handle variable workloads, allowing organizations to scale their AI capabilities as needed</li><li>Reduced costs: When usinAIaaS, businesses can only pay for the services they use, without the need for new investments in additional work hours and/or any hardware equipment</li><li>Experiment: Companies can experiment with different AI technologies and easily switch between them as requirements change. No need to tie oneself with any particular solution</li><li>Race to the market: Pre-trained AI models and services and help companies to implement their solutions and deliver them to the target audiences much faster</li><li>Access to new technologies: AIaaS is something that is still being carefully explored and continuously developed and any company can have the front-row seat to new updates and solutions emerging from AIaaS providers</li></ul><p>AIaaS works by offering AI capabilities through APIs or web interfaces, which developers can use to integrate new AI functionalities into their applications and services. AIaaS providers typically offer pre-made AI models, algorithms, and tools, which have been trained on versatile and vast amounts of data. Organizations can also train custom models using their own data, depending on the service provider capabilities. In the past few years, we witnessed AIaaS solutions slowly growing their market presence, fueled by the increased demand for AI solutions and the maturity of AI technologies. We are far of the old language models and flaky AI solutions, aren’t we? Some major cloud providers, such as <a href="https://cloud.google.com/products/ai">Google Cloud AI</a>, <a href="https://azure.microsoft.com/en-us/solutions/ai/">Microsoft Azure AI</a>, and <a href="https://aws.amazon.com/machine-learning/ai-services/">Amazon Web Services (AWS) AI</a>, of course make a large portion of the existing AIaaS landscape with their wide range of AI services and tools. In addition to these large providers, there are numerous specialized AIaaS companies targeting specific industries or use cases, such as healthcare, finance, and marketing.</p><h4>Common Misconceptions</h4><p>As with any new or upcoming technology or solution, there are bound to be some misconceptions surrounding them. Misconceptions regarding some technologies can take on a life of their own and spread beyond the fenced safety of <em>uninformedy</em>. We can only remember situations like the <a href="https://en.wikipedia.org/wiki/Year_2000_problem">Y2K</a> problem. So, here are some of the misconceptions surrounding AIaaS:</p><ul><li>AIaaS is here to replace human expertise: This is common one whenever AI, in general, is mentioned, isn’t it? AIaaS is mainly designed to support and enhance developer experience, not replace it. While AI can automate repetitive tasks and provide valuable insights, it still requires a level of human interaction and guidance in order to make critical decisions and ensure ethical considerations are addressed. AI models can sometimes produce unexpected or biased results, and human oversight is necessary to ensure that AI-driven decisions align with company goals and values.</li><li>AIaaS can replace data scientists: This is a continuation on the previous point, and everything mentioned there can be applied in this case as well. While AIaaS simplifies the process of integrating AI capabilities into various apps, data scientists still need to claim an important role in the development and implementation of those solutions. Data scientists have the needed domain knowledge and expertise to choose the right algorithms, models, and data analytics tools to interpret the obtained data. This is essential for the success of any AI projects. While AIaaS can streamline some of the the work currently occupied by data scientists, it does not replace their knowledge and judgment.</li><li>Everything can be achieved with one solution: AIaaS platforms offer a wide variety of AI tools and services designed to cater to different industries and use cases. Every company should carefully evaluate their specific requirements and choose the appropriate AIaaS service that meets their needs. The most common scenario could be tailoring and customizing of AI models and algorithms or joining multiple AI services in order to deliver on the requirements set by some use case.</li><li>Everything is a security risk: While AIaaS does introduce new security concerns, many AIaaS providers place a strong emphasis on security and privacy of their services. Providers tend to follow industry best practices in securing their infrastructure and protecting User data. Since everything about this technology is new, mistakes are most likely to manifest themselves from time to time, but each and every mistake should lead to more precise improvements and less errors in the future. Companies are presented with security measures implemented by AIaaS providers and it’s up to them to analyze and choose those that meet their security criteria. It’s always recommended to use additional, abstract security layers in order to have a bit more confidence that the security measures in place will suffice.</li></ul><p>These are just some of the most saturated misconceptions surrounding AIaaS at the time of this writing, and by looking a bit deeper into them, we can better understand the true potential of this new tool and make informed decisions for all of our existing or upcoming projects when regarding the integration with any AIaaS.</p><h4>Use Cases in the Industry</h4><p>While it might be early to form an extensive case study of the whole AIaaS ecosystem since it might not be seen as a go-to solution by many, there are existing use cases and examples where this technology has already made strides in the industry. These are just some of the examples where existing AI solutions were utilized in order to enhance the overall product in some kind of way.</p><ul><li>Medicine and healthcare: Early on, it was shown that AI has the needed potential to revolutionize healthcare by enabling the development of diagnostic tools, treatment plans, and remote patient monitoring solutions. Some AIaaS medical image analysis services can help in identifying diseases and anomalies in medical images, such as X-rays, MRIs, and CT scans. Companies like <a href="https://www.crunchbase.com/organization/zebra-medical-vision">Zebra Medical Vision</a> provide AIaaS solutions specifically tailored for healthcare applications.</li><li>ChatBots: What company doesn’t use some form of ChatBots today? OK, this might be an exaggeration, but <em>chatbots</em><strong><em> </em></strong>seem to be everywhere now, don’t they? Nevertheless, AIaaS can be used to create <em>intelligent </em>chatbots for handling customer inputs and queries, provide personalized support and even automate tedious tasks. Natural language processing (NLP) and machine learning (ML) services can be integrated into chatbots in order to better understand customer queries, generate responses, and even advance complex issues to human company personnel. Google, for example, has its own solution for a very true-to-life chatbot in the form of <a href="https://cloud.google.com/dialogflow">Dialogflow</a>,and it can be easily integrated with existing customer support systems.</li><li>Content writing and content moderation: I’ve already used, for example, OpenAI’s <a href="https://openai.com/blog/chatgpt">ChatGpt</a> service in various of ways, and to even enhance my own <a href="https://sdremthix.medium.com/discovering-quantum-computing-bb8b4e53e74d">articles</a>. This only shows that AIaaS can be utilized for automating content writing and moderation on social media platforms, comment boards, and websites, making sure that any user-generated content adheres to community guidelines and policies. There are AI content moderation services that can with a great deal of success analyze text patterns, images, and even video content in order to identify and filter out inappropriate content, based on whatever criteria is needed of it. Google <a href="https://cloud.google.com/vision/docs/detecting-safe-search">Cloud Vision</a> is one such service that offers pre-built content moderation and it can be integrated into existing products. On the other hand, <a href="https://www.jasper.ai/">JasperAI</a> is one AIaaS platform that offers content creation and copywriting services tailored to any company’s needs that is less likely to be detected as AI-generated content by other AIaaS’s.</li><li>Data analytics: AI solutions can be used to extract valuable insights from large volumes of data, enhancing the decision-making and overall business performance. Data processing can be automated with producing shorter feedback loops when trying to identify trends and patterns on the market and potentially predict future outcomes.</li><li>Marketing to a target audience: It can be very tedious to tailor content specifically made for each and every costumer of our services, bit that’s required nowadays in order to stay competitive on the market. AI-driven solutions can help and enhance the marketing jobs by providing personalized customer experiences, targeted advertising, and behavior analysis. AIaaS-based NLP and ML services can analyze customer data in order to identify preferences, predict behavior, and create tailored marketing campaigns (and content). IBM <a href="https://www.ibm.com/watson/customer-engagement/resources/watson-campaign-automation/">Watson Campaign Automation</a> is one such solution, tailored for businesses to step up their marketing game.</li><li>Quality control: Improving manufacturing processes and quality control can be a difficult enough task, but by automating anomaly detection, optimizing task scheduling, and predicting machine failures with the usage of AI, we can definitely mitigate those concerns. The cost-to-effectiveness can be proven significant. Computer vision services can be used to scan different products for defects, while ML services can be used in the same way to analyze various sensor data to identify patterns and anomalies that may point to potential (upcoming) issues. As an example, <a href="https://sightmachine.com/">Sight Machine</a> offers specialized solutions for various industries by transforming the manufacturing processes.</li></ul><p>These are just some of the existing use cases where various AIaaS solutions have been implemented and where the same solutions have enhanced existing work pipelines. The potential is definitely there to introduce this new technology and services into other fields.</p><h4>A Practical Example? Why not?</h4><p>We already covered some of the basis surrounding AIaaS and those familiar with <a href="https://azure.microsoft.com/en-us/resources/cloud-computing-dictionary/what-is-saas">SaaS</a> (software as a service), pretty much know what this new AI provisioning is all about. But, can we showcase an example AIaaS solution with real, down-to-earth, technologies and frameworks, one of our own liking? Let’s see what we can do, shall we?</p><p><strong>Problem Statement</strong></p><p>First off, we need a valid use case for our AIaaS solution and since this is only an introduction article about AIaaS, we can focus on the vary basics. In this section we’ll design a profanity filter for User submitted textual data. I should note that this won’t be a step-by-step tutorial.</p><p><strong>Backend</strong></p><p>Our backend will be built around <a href="https://www.python.org/">Python</a>, <a href="https://pytorch.org/">PyTorch</a> and <a href="https://flask.palletsprojects.com/en/2.3.x/">Flask</a>. We’ll train our profanity model using bi-directional long-short term memory (<a href="https://www.researchgate.net/figure/Architecture-of-the-bidirectional-long-short-term-memory-BiLTSM-conditional-random_fig3_345363169">BiLTSM</a>) neural network on a very plain CSV dataset. It should be noted that the success and accuracy of our profanity filter detection service greatly relies on the size and variety of the dataset on which the model will be trained on. The dataset contains only two columns, “<em>text</em>” and “<em>label</em>” for matching a given String data to the Boolean flag (0,1).</p><p>To get started, let’s make sure that we have the right dependencies installed on our host machine:</p><pre>pip3 install Flask flask-cors torch torchtext spacy pandas sklearn<br>python3 -m spacy download en_core_web_sm</pre><p>With all dependencies installed, we can proceed with creating our first python class for pre-processing the given <strong>dataset.csv</strong> file:</p><pre>import pandas as pd<br>from sklearn.model_selection import train_test_split<br>from torchtext.data.utils import get_tokenizer<br>from torchtext.vocab import build_vocab_from_iterator<br>import torch<br>import spacy<br>from torch.utils.data import Dataset<br><br>class TextDataset(Dataset):<br>   def __init__(self, data, tokenizer, vocab):<br>       self.data = data<br>       self.tokenizer = tokenizer<br>       self.vocab = vocab<br><br>   def __len__(self):<br>       return len(self.data)<br><br>   def __getitem__(self, idx):<br>       text, label = self.data.iloc[idx]<br>       tokens = self.tokenizer(text)<br>       token_indices = [self.vocab[token] for token in tokens]<br>       return torch.tensor(token_indices), torch.tensor(label)<br><br>def yield_tokens(data_iter):<br>   tokenizer = get_tokenizer(&quot;spacy&quot;) #en_core_web_sm<br>   for _, row in data_iter:<br>       yield tokenizer(row[&#39;text&#39;])<br><br>nlp = spacy.load(&quot;en_core_web_sm&quot;, disable=[&quot;tagger&quot;, &quot;parser&quot;, &quot;ner&quot;])<br><br><br>def preprocess_text(text, vocab):  # add vocab as an argument<br>   tokenized_text = [token.text for token in nlp(text)]<br>   numerized_text = [vocab[token] for token in tokenized_text]<br>   return numerized_text<br><br>def load_dataset(file_path):<br>   data = pd.read_csv(file_path)<br>   train_data, test_data = train_test_split(data, test_size=0.2, random_state=42)<br><br>   tokenizer = get_tokenizer(&quot;spacy&quot;)<br>   vocab = build_vocab_from_iterator(yield_tokens(train_data.iterrows()))<br><br>   train_dataset = TextDataset(train_data, tokenizer, vocab)<br>   test_dataset = TextDataset(test_data, tokenizer, vocab)<br><br>   return train_dataset, test_dataset, len(vocab), vocab<br><br>if __name__ == &quot;__main__&quot;:<br>   train, test, vocab_size, vocab = load_dataset(&quot;dataset.csv&quot;)<br>   print(f&quot;Vocabulary size: {vocab_size}&quot;)</pre><p>With the <strong>preprocess.py</strong> file in place, we need to create the actual model. We proceed with creating a new <strong>model.py</strong> file:</p><pre>import torch<br>import torch.nn as nn<br><br>class BiLSTM(nn.Module):<br>   def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, num_layers, dropout):<br>       super(BiLSTM, self).__init__()<br><br>       self.embedding = nn.Embedding(vocab_size, embedding_dim)<br>       self.rnn = nn.LSTM(embedding_dim, hidden_dim, num_layers, bidirectional=True, dropout=dropout, batch_first=True)<br>       self.fc = nn.Linear(2 * hidden_dim, output_dim)<br>       self.dropout = nn.Dropout(dropout)<br><br>   def forward(self, text):<br>       embedded = self.dropout(self.embedding(text))<br>       output, (hidden, _) = self.rnn(embedded)<br>       hidden = self.dropout(torch.cat((hidden[-2, :, :], hidden[-1, :, :]), dim=1))<br>       return self.fc(hidden)</pre><p>The <strong>BiLSTM</strong> class inherits form the <strong>nn.Module </strong>and uses the <strong>nn.Embedding</strong> for creating new vectors. We use the <strong>nn.Linear </strong>class to apply linear transformation and create a new connecting layer. At the end, we apply the <strong>nn.Dropout</strong> for regularizing and preventing <a href="https://machinelearning.wtf/terms/co-adaptation/">co-adaptation</a> in the neural network. This is almost certainly required since we have a small dataset to train our model with.</p><p>The next file we need to create is <strong>train.py</strong>:</p><pre>import torch<br>import torch.nn as nn<br>from torch.nn.utils.rnn import pad_sequence<br>import torch.optim as optim<br>from torch.utils.data import DataLoader<br>from model import BiLSTM<br>from preprocess import load_dataset<br><br><br>def train(model, iterator, optimizer, criterion):<br>   model.train()<br>   epoch_loss = 0<br><br>   for batch in iterator:<br>       text, label = batch<br>       text, label = text.to(device), label.to(device)<br>       optimizer.zero_grad()<br>       predictions = model(text).squeeze(1)<br>       loss = criterion(predictions, label.float())<br>       loss.backward()<br>       optimizer.step()<br>       epoch_loss += loss.item()<br><br>   return epoch_loss / len(iterator)<br><br><br>def custom_collate(batch):<br>   texts, labels = zip(*batch)<br>   padded_texts = pad_sequence([torch.tensor(t) for t in texts], batch_first=True)<br>   labels = torch.tensor(labels)<br>   return padded_texts, labels<br><br><br>if __name__ == &quot;__main__&quot;:<br>   device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)<br><br>   train_data, test_data, vocab_size, vocab = load_dataset(&quot;dataset.csv&quot;)<br><br>   batch_size = 32<br>   train_iterator = DataLoader(train_data, batch_size=batch_size, shuffle=True, collate_fn=custom_collate)<br>   test_iterator = DataLoader(test_data, batch_size=batch_size, shuffle=False, collate_fn=custom_collate)<br><br>   embedding_dim = 100<br>   hidden_dim = 64<br>   output_dim = 1<br>   num_layers = 2<br>   dropout = 0.5<br><br>   model = BiLSTM(vocab_size, embedding_dim, hidden_dim, output_dim, num_layers, dropout).to(device)<br><br>   optimizer = optim.Adam(model.parameters())<br>   criterion = nn.BCEWithLogitsLoss()<br><br>   num_epochs = 10<br><br>   for epoch in range(num_epochs):<br>       train_loss = train(model, train_iterator, optimizer, criterion)<br>       print(f&quot;Epoch {epoch+1}/{num_epochs}, Loss: {train_loss:.4f}&quot;)<br><br><br>   torch.save(model.state_dict(), &quot;profanity_model.pt&quot;)</pre><p>We train the new model for one <em>epoch </em>(one iteration through the dataset), define the parameters needed for the <strong>BiLSTM </strong>and finally save the newly trained model into a new <strong>.pt </strong>file.</p><p>To train the model, we need to be sure that we have a <strong>dataset.csv </strong>file in our project directory at root level and that all of the dependencies have been installed. Now we can train our model by executing the following command:</p><pre>python3 train.py</pre><p>If we’ve done everything correctly, we should now have a <strong>profanity_model.pt </strong>file in our project directory and we can continue our AIaaS solution by defining a simple REST service.</p><p>For the API layer, we’ll utilize Flask and create a simple controller and one POST endpoint <strong>/check</strong>. We can now proceed by creating an entry-point <strong>app.py</strong> file:</p><pre>import torch<br>from model import BiLSTM<br>from preprocess import preprocess_text, load_dataset<br>from flask import Flask, request, jsonify<br>from flask_cors import CORS<br><br># Load the dataset to get the vocab object<br>_, _, _, vocab = load_dataset(&quot;dataset.csv&quot;)<br><br># Define the model parameters<br>vocab_size = len(vocab)<br>embedding_dim = 100<br>hidden_dim = 64<br>output_dim = 1<br>num_layers = 2<br>dropout = 0.5<br>device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)<br><br># Profanity threshold<br>threshold = 0.5<br><br>app = Flask(__name__)<br>CORS(app)<br><br># Load the trained model<br>model = BiLSTM(vocab_size, embedding_dim, hidden_dim, output_dim, num_layers, dropout)<br>model.load_state_dict(torch.load(&quot;profanity_model.pt&quot;))<br>model.eval()<br><br>@app.route(&quot;/check&quot;, methods=[&quot;POST&quot;])<br>def check():<br>   if not request.json or &quot;text&quot; not in request.json:<br>       return jsonify({&quot;error&quot;: &quot;Missing &#39;text&#39; key in the JSON request object.&quot;}), 400<br>  <br>   text = request.json[&quot;text&quot;]<br>   preprocessed_text = preprocess_text(text,vocab)<br>   tensor_text = torch.tensor([preprocessed_text]).to(device)<br>   output = model(tensor_text).squeeze(1)<br>   prob = torch.sigmoid(output).item()<br><br>   contains_profanity = prob &gt; threshold<br><br>   return jsonify({&quot;profanity_probability&quot;: prob, &quot;contains_profanity&quot;: contains_profanity})<br><br>if __name__ == &quot;__main__&quot;:<br>   app.run(debug=True)</pre><p>In this simple Flask controller, we are importing the previously created model class and preprocessor functions, load our new vocabulary and define a threshold of accuracy (we are receiving float values as return statements from our model) for the generated output values. We then load our newly created model and enable the evaluation state. We define a new <strong>check</strong> endpoint function and method (POST) by utilizing a new Flask app instance. In the function body we check for the request JSON object validity, extract any textual data, check the vocabulary for the existence of the said data and then pass it to our model, validate the probability for any profanity detected and output the result into a new JSON object.</p><p>To run our service we need to execute the following command:</p><pre>python3 app.py</pre><p>Our service should be up and running on <strong>localhost:5000</strong> and we can make a simple POST request using the following JSON object in the request body:</p><pre>{<br> &quot;text&quot;: &quot;Check me for any profanity.&quot;<br>}</pre><p>Just to be complete, we can also execute the following <a href="https://curl.se/">curl</a>:</p><pre>curl -X POST -H &quot;Content-Type: application/json&quot; -d &#39;{&quot;text&quot;: &quot;text&quot;: &quot;Check me for any profanity.&quot;<br>}&#39; http://127.0.0.1:5000/check</pre><p><strong>Frontend</strong></p><p>For the frontend component of our new service, we chose the <a href="https://react.dev/">React</a> framework to help us with quickly setting up a new client application. For the next steps, we need to make sure that we’ve installed <a href="https://nodejs.org/en">NodeJS</a> and npm on our workstations.</p><p>We can create a new React client application by running the <strong>create-react-app </strong>command:</p><pre>npx create-react-app profanity_filter_client</pre><p>After the new React application has been created, we can verify that everything is setup correctly by running <em>npm start</em>. Our React template app should be available on <a href="http://localhost:3000/"><strong>http://localhost:3000/</strong></a><strong> </strong>(make sure no other service is running on this port on your machine).</p><p>Next we’ll install <a href="https://axios-http.com/docs/intro">Axios</a> as our networking client:</p><pre>npm install axios</pre><p>We can now create our new <strong>ProfanityFilter.js</strong> component:</p><pre>import React, { useState } from &#39;react&#39;;<br>import axios from &#39;axios&#39;;<br><br>const API_ENDOINT = &#39;http://127.0.0.1:5000/check&#39;;<br><br>const ProfanityFilter = () =&gt; {<br>   const [inputText, setInputText] = useState(&#39;&#39;);<br>   const [hasProfanity, setHasProfanity] = useState(null);<br>   const [isLoading, setIsLoading] = useState(false);<br>   const [error, setError] = useState(null);<br><br>   const checkProfanity = async (text) =&gt; {<br>       try {<br>           setIsLoading(true);<br>           const response = await axios.post(API_ENDOINT, {<br>               text: text,<br>           });<br>           setHasProfanity(JSON.parse(response.data.contains_profanity));<br>       } catch (err) {<br>           setError(&#39;Problem executing API call!&#39;);<br>       } finally {<br>           setIsLoading(false);<br>       }<br>   };<br><br>   const handleSubmit = (e) =&gt; {<br>       e.preventDefault();<br>       checkProfanity(inputText);<br>   };<br><br>   return (<br>       &lt;div&gt;<br>           &lt;h1&gt;Profanity Filter&lt;/h1&gt;<br>           &lt;form onSubmit={handleSubmit}&gt;<br>               &lt;div&gt;<br>                   &lt;label htmlFor=&quot;input-text&quot;&gt;Enter text to be checked:&lt;/label&gt;<br>                   &lt;textarea<br>                       id=&quot;input-text&quot;<br>                       value={inputText}<br>                       onChange={(e) =&gt; setInputText(e.target.value)}<br>                   /&gt;<br>               &lt;/div&gt;<br>               &lt;button type=&quot;submit&quot;&gt;Check Profanity&lt;/button&gt;<br>           &lt;/form&gt;<br><br>           {isLoading &amp;&amp; &lt;p&gt;Loading...&lt;/p&gt;}<br>           {error &amp;&amp; &lt;p&gt;{error}&lt;/p&gt;}<br>           {hasProfanity !== null &amp;&amp; (<br>               &lt;p&gt;<br>                   {hasProfanity<br>                       ? &#39;Text contains profanity!&#39;<br>                       : &#39;Text does not contain profanity.&#39;}<br>               &lt;/p&gt;<br>           )}<br><br>       &lt;/div&gt;<br>   );<br>};<br><br>export default ProfanityFilter;</pre><p>In this component we firstly import the necessary dependencies and define our service target API_ENDPOINT (make sure that our backend service is running). Then we create a new functional React component with two functions: <em>checkProfanity </em>and handleSubmit. The <em>checkProfanity </em>function makes the API call to our backend service and passes the input data we’ll provide as our input data. We display a simple <em>isLoading</em> message while we wait for the response data to arrive and set the response value in our <em>hasProfanity</em> state. Based on the response that we receive, two messages can be displayed.</p><p><strong>Deployment and Scaling</strong></p><p>Containerization via <a href="https://www.docker.com/">Docker</a> and <a href="https://kubernetes.io/">Kubernetes</a> or similar tools can be a valid approach in deploying our backend and frontend components onto dedicated hosting platforms. We can also utilize specialized platforms like <a href="https://www.heroku.com/">Heroku</a> or <a href="https://www.netlify.com/">Netlify</a> for hosting our backend and frontend components individually. These platforms can help us with monitoring, scaling, load balancing, and logging of our services. We can also significantly improve the performance and response time of our solutions with implementing some basic or advanced caching mechanisms. In-Memory data stores like <a href="https://redis.io/">Redis</a> can help us achieve our performance goals.</p><p>It should be noted that this is a very rudimental example of what an AIssS solution can be and there’s a lot of room for improvement and expansion.</p><p>I’ve intentionally omitted the whole end-to-end solution details such as authentication, rate limiting, error handling, internationalization, styling, <a href="https://www.databricks.com/glossary/mlops">MLOps</a> and tests for the sake of brevity. These all can be subject for future articles on their own.</p><h4>Best Practices</h4><p>Like most software solutions, the best practices surrounding AIaaS are techniques that cascade through all the different areas of software development. Techniques like <a href="https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html">screaming</a> architecture and organized codebase, form the very foundation that we’ll base our new and interesting (preferably profitable too) solutions on. We can organize our codebase by individual features or by type, but no matter what approach we decide to go with, we should aim to consider clear separation of concerns. What does that mean? It means that we shouldn’t mix and match different types of components and classes or functions in huge blobs of code. Each type of component should have their own place in the project structure and their own package/folder to reside in. When two or more components of different types need to communicate with each other we can use dependency injection (<a href="https://en.wikipedia.org/wiki/Dependency_injection">DI</a>) along with composition to satisfy the component relationship requirements.</p><p>Other notable practices we should adhere to can be of the following:</p><ul><li>Shared code style and commit message rules</li><li>Use virtual environments to avoid dependency conflicts between two or more projects</li><li>All sensitive information (secrets,API keys, passwords, etc.) should stay off any version control system (Git, <a href="https://subversion.apache.org/">Subversion</a>, <a href="https://en.wikipedia.org/wiki/Mercurial">Mercurial</a>). Preferably, all of that information should stay in environment configurations or special vaults when using different online repositories</li><li>Logging and handling of exceptions</li><li>Input validations across all layers of the system. This can be enforced by using domain layer rules</li><li>Standardized http status codes in our API responses also leak prevention of the underlying platform (stack traces)</li><li>Unit, integration and end-to-end tests</li><li>Following the DRY (do not repeat yourself) and KISS (keep it simple and straightforward) principles</li></ul><p>If we try to follow these best practices, we can truly build a well-structured and maintainable AIaaS solution. This will make it easier to scale any application, implement new features, and collaborate with other team members, ensuring that this kind of solution keeps it’s focus on the problem domain rather than combating spaghetti code or endless time in searching for underlying issues.</p><h3>Architectural Recommendations</h3><p>Depending on the company’s needs and long-term projection goals for a AIaaS solution, different architectures can be utilized for creating and setting an easily maintainable environment for the new product. Let’s take a look at some of the different architecture and patters at our disposal:</p><p><strong>Clean Architecture</strong></p><p>Those who follow my articles and posts, realize that I like to talk about <a href="https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html">clean architecture</a> a lot. I’m certainly no expert in this field. What always fascinated me about clean architecture is its abstract nature. There is a simple rule, the dependency direction, at that’s basically it. But by trying to implement any solution around that simple rule, can be challenging to say the least, and if we try to migrate existing solutions to a clean architecture, the development pain gets higher and higher.</p><p>Just to be complete, let’s look what clean architecture means. Clean architecture is mostly a design principle that focuses on separation of concerns, maintainability, and testibility at its core. Each system layer has its own responsibility and role in the overall solution. For AIaaS solutions, we can apply clean architecture by separating the AI model, data processing, presentation and API logic into distinct layers all of which should adhere to a common business/domain rules and contracts layer. This ensures that changes in one layer have minimal to no impact on others, making it easier to update or extend your solution.</p><p><strong>Microservices Architecture</strong></p><p>The <a href="https://microservices.io/patterns/microservices.html">microservices architecture</a> focuses around developing and deploying small, independent services that communicate with each other. The main concept around this architecture is the loose coupling between individual services, allowing each service to be developed, and deployed independently. In the context of AIaaS, we can create separate services for different AI models or functionalities and scale them independently based on business requirements.</p><p><strong>Event-driven Architecture</strong></p><p>An event-driven architecture focuses on producing, detecting, and reacting to events fired by different components in the system. Services communicate asynchronously using events, which can be triggered by changes made to the global system state. Other components subscribe to the events needed for their functionalities and react and update their individual states based on the defined criteria. This has a huge impact in the overall system performance and responsiveness, as well as the promotion of decoupling between components. AIaaS solutions can use this type of architecture in order to handle asynchronous processing, such as updating AI models or processing large datasets.</p><p><strong>Serverless Architecture</strong></p><p>I have to admit, I have the least experience with this type of architecture, but when talking about AIssS it’s nonetheless one of the more important architecture principles. A <a href="https://aws.amazon.com/lambda/serverless-architectures-learn-more/">serverless architecture</a> allows us to focus on writing code without having to worry about any infrastructure management. With serverless, we can deploy functions that automatically scale based on demand. The business model around this is to pay only for the resources we already used. This can help reduce operational costs and significantly improve scalability of systems. There are certain pitfalls to this, with some cases of high resource usage spikes that reflect on the monthly bills and as such we have to be careful how we implement our solutions. In AIaaS we can use serverless platforms like <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> or <a href="https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview">Azure Functions</a> to deploy our AI models and services.</p><p><strong>API Gateways</strong></p><p>In the <a href="https://learn.microsoft.com/en-us/dotnet/architecture/microservices/architect-microservice-container-applications/direct-client-to-microservice-communication-versus-the-api-gateway-pattern">API gateway</a> solutions we define a single entry point for managing and routing API requests to different microservices. This approach simplifies client interactions by providing a single interface, handling request routing, and receiving responses from multiple services. If our AIaaS solutions revolves around different microservices, we can use an API gateway to manage access control, load balancing, and caching.</p><p>It can be a bit ungrateful talking about architecture recommendations for different software solutions since they vastly depend on the use cases we require. These are just some example architectures, and not nearly all of them, that we can use to structure our AIaaS solutions around. We are also not limited to a single architecture for our solutions. Mixing and matching different architecture for different areas of our solutions can provide a significant stronghold for a system that is adaptable to upcoming market requirements. If we go with the multi-architecture approach, we should consider the maintainability and scalability of such solutions since we are adding additional complexity to them for which additional resources might be required. If this peaks your interest, I recommend looking into more advanced software architectures that could possibly be a better fit for whatever solution you had in mind.</p><h4>Future of AIaaS</h4><p>AI seems to be here to stay, at least for a while and it might be safe to say that we’ve only scratched the surface when it comes to the potential for this amazing new tool. As software developers we need to make informed decisions regarding future project roadmaps. AIaaS has the potential to display AI to a broader range of Users with some ease-of-access and ease-of-use techniques. Bringing down the unknown concepts surrounding this technology and making it understandable to a wide audience, can help the process of advancing AI to new, as of yet, unimaginable levels. In addition, AIaaS with affordable pricing models can make it more accessible to small businesses and individual Users.</p><p>In the world of IoT (internet-of-things) and edge computing, AIaaS solutions can provide large advantages. Specifically trained AI models for edge devices, can help in improving latency, minimize data transfers (and costs) and advance real-time processing. Certain trends can emerge in which specialized IoT devices for smart cities, help in managing power output, traffic data and waste management. AIaaS can also help in improving in the effectiveness of predicting device sensor failures and even schedule maintenance service where required. We shouldn’t also skip over the various health and wearable device data that could potentially improve the greater wellbeing of patients.</p><p>At the time of of writing this article, a few concerns have popped-up regarding AI and specific language models. One of which is undoubtedly plagiarism. Naimley, the most popular AI tools out there, use specific language models, and to be extremely crude in explaining now, ‘splice’ together various data on which the various models have been trained on, in order to create an global illusion of consciousness. A large portion of the current AI tools out there have copious amounts of data that they combine and form new, coherent data, that is context-sensitive and relevant to the request input data. The more prevalent an issue is documented in the training data, the more precise and relevant the output data produced by the AI tools is. This has a huge advantage of the current search engines since context is everything nowadays. Since the training data out of which the AI models are based on, had to come out of somewhere and not just materialize itself out of nowhere, we are witnessing large amount AI-produced content that seems to be way too similar to existing articles, codebases, books and other copyright-protected material that is stored somewhere. A large number of content creators out there, had put significant amounts of time and effort into creating their work, only for it to be fed into training models and further passed on as seemingly original content for other Users inquiries. This is definitely a two-sided coin, since the larger the input data for the models, the more unique and exact output will be produced, on the other side is intellectual property. There a certainly way around this. A lot of companies that specialize in various fields, train their AI models on their own data that is composed from years of working in the industry. AI models trained in such a way, are transparent to other intellectual properties.</p><p>AI models can also be biased! Yes, that’s a thing nowadays. There can also be a layer of discrimination if a model is trained on biased data. Look no further than our little profanity filter example from earlier. Who says what ‘profanity’ is? We can train our model in any direction we want to. AIaaS providers should be aware of potential biases when developing their AI solutions with certain ethical boundaries in place. There are already solutions or at least techniques, bespoke in addressing these potential issues. One such technique is <a href="https://www.ibm.com/watson/explainable-ai">explainable AI</a> (XAI) that aims to de-structure the process of AI model ‘reasoning’ and analyze what goes into making a model more accurate and transparent. I’m certainly not here to pass any judgment on this subject, but mostly to showcase a potential issue when using specific AI models for the future development of new solutions.</p><p>We’ve only touched upon this in the previous paragraphs but we have seen, AIaaS solutions often process sensitive data, and it’s important to also implement a different kind of security measures. In AIaaS solutions, providers need to address issues surrounding data storage, data migrations, and authorizations. Data encryption, access roles and regular security audits can help address potential risks.</p><h4>Final Notes</h4><p>Many companies have already recognized the potential in AIaaS, and by <em>jumping onto the bandwagon</em> early on, managed to secure their place as providers of knowledge and experience for others who are about to join them.</p><p>AI as a Service (AIaaS) can be a powerful tool for organizations to make use AI capabilities without ever having to invest into their own development and research. It’s recommended for sure, but not necessary. Also the costs of infrastructure and trainings can be significantly reduced by delegating such tasks to existing providers on the market. With AIaaS, AI solutions are made more accessible, scalable and affordable. In this article, I tried to present this new concept, showcase an example solution along with a few use cases, debunk common misconceptions and even dance a bit with future ideas and concerns.</p><p>What are your thoughts on AIaaS? As always, your feedback and thoughts are greatly valued, so feel free to leave a comment, question or message me. Only with your input and feedback can I improve the quality of this kind of content.</p><p>The complete source code for the <em>profanity filter </em>project discussed in this article can be found on <a href="https://github.com/Remthix/ml-profanity-filter-AIaaS">here</a>.</p><p>Thank you for reading, and happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c33b799b2a6c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Programming to an Interface]]></title>
            <link>https://sdremthix.medium.com/programming-to-an-interface-1ad65d437f97?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ad65d437f97</guid>
            <category><![CDATA[objectorientedprogramming]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[design-patterns]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 24 Apr 2023 07:07:26 GMT</pubDate>
            <atom:updated>2023-05-05T11:21:35.545Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fPuibpn20dkDGj7UQ0Hrrw.png" /></figure><h4>Intro</h4><p>How often do we as software developers hear the words ‘maintainable’, ‘scalable’ and ‘testable’ when we talk about software solutions? I’ll venture a guess, <em>a</em> <em>lot</em>? And to an extend, how often is it that the proposed answers are murky or overly complex at best? Unlike the previous question, I’ll leave the answer to You, the reader. The one thing I could possibly contribute is one of the more important techniques in object oriented programming (OOP) and that is: <em>programming to an interface</em>.</p><p>A lot of object-oriented programming centers around the design principles of programming to an interface with it defining the behavior of an <em>object</em> through an interface (a collection of abstract methods or properties) rather than depending on some concrete implementation of the said interface.</p><p>Programming to an interface can provide a variety of benefits in many software solutions, including:</p><ul><li><strong>Encapsulation</strong>: By defining a contract through an interface, you can hide the details of an object’s implementation (implementation detail). This level of abstraction focuses only on the functionality that is needed, leaving any internal state and behavior of an object protected.</li><li><strong>Testability</strong>: Programming to an interface enables easier testing, as you can create <em>mock</em> implementations of the interface for unit testing. This can be extremely powerful since it enables testing certain components in isolation.</li><li><strong>Decoupling</strong>: Using contracts for the purpose of communication between two or more components in a project can make it easier to modify, extend, or even replace entire components, the implementation detail at least, without affecting other dependant components. Needless to say that this can be a large contributor to making more modular and maintainable solutions.</li><li><strong>Flexibility</strong>: Hidden behind interface contracts, third-party solutions and libraries can be easily replaced with other ones. The only important thing is that the interface contracts are fulfilled by the new libraries. This is particularly useful when working with third-party libraries or APIs, as it enables you to adapt your code to new requirements or updates without major refactoring.</li><li><strong>Reusability</strong>: When you program to an interface, you can reuse code that depends on the interface with any object that implements it.</li></ul><p>For this article we’ll try to together explore the concept of programming to an interface. The examples provided are written using the <a href="https://kotlinlang.org/">Kotlin</a> programming language since it’s a modern programming language for the JVM that has become increasingly popular over the past few years. It’s perfectly alright If you might not be not familiar with the Kotlin programming language, since most of the example concepts given can be applied to any programming language that has an contract abstraction feature that supports the ideas behind programming to an interface.</p><h4>Again, why Kotlin?</h4><p>Because I needed the refresher on the programming language at the time of writing this article. Other than that, Kotlin is a modern, statically-typed programming language that runs on the Java Virtual Machine (JVM). The language has gained in popularity and usage due to its null safety features, modern syntax, and interoperability with the Java programming language. The language has gained a lot of boost when it was officially supported by Google (Alphabet) for Android app development.</p><p>In Kotlin, interfaces play a crucial role in supporting the programming to an interface design principle. Kotlin interfaces are similar to Java interfaces. Like Java, In Kotlin interfaces can have default method implementations, which can help reduce code duplication and simplify inheritance structures. Kotlin interfaces allow you to define a contract that classes must adhere to when implementing the interface. They consist of abstract methods and properties, which must be overridden and implemented by the implementing classes.</p><p>In the following sections, we will delve into the details of defining, implementing, and working with interfaces in Kotlin, providing examples and use cases to showcase the power of programming to an interface in this modern programming language.</p><h4>Interfaces explained</h4><p>In software development, interfaces are abstract types that define a contract or a set of rules for classes that implement them need to follow (hence <em>contract</em>). Interfaces usually declare method signatures and properties which the implementing classes must provide definitions for.</p><p>In Kotlin, we can define an interface using the <strong><em>interface</em></strong> keyword, followed by the interface name and an optional list of supertypes. Inside the interface, we can declare abstract properties, methods, and even provide default implementations for some methods.</p><pre>interface InterfaceName {<br>    // Methods and properties<br>}</pre><p>A simple interface with methods and properties can look something like the following:</p><pre>interface Shape {<br>    val name: String<br>    fun area(): Double<br>    fun perimeter(): Double<br>}</pre><p>In the example above the <strong>Shape</strong> interface has two abstract methods, <strong>area</strong> and <strong>perimeter</strong> and one property, <strong>name</strong>.</p><p>With a defined interface, we can proceed with the process implement it in a class. We will use a colon (<strong>:</strong>) after the class name, followed by the name of the interface:</p><pre>class ClassName : InterfaceName {<br>    // Class implementation<br>}</pre><p>When implementing an interface, we must provide implementations for all abstract methods and properties declared in the interface itself. We do that by using the keyword <strong>override </strong>to explicitly note that we are providing an implementation for an interface method or property. If we use our previous <strong>Shape</strong> interface example onto a new implementing class, ti can look something similar to this:</p><pre>class Circle(val radius: Double) : Shape {<br>    override val name: String = &quot;Circle&quot;<br><br>    override fun area(): Double {<br>        return Math.PI * radius * radius // Math is a standard class in the java.lang package<br>    }<br><br>    override fun perimeter(): Double {<br>        return 2 * Math.PI * radius<br>    }<br>}</pre><p>The <strong>Circle </strong>class above needs to fulfill the interface contract in order to satisfy the Kotlin compiler. It should be noted that this can also be done without the <em>actual </em>implementation method contents and easily be replaced with <strong>TODO </strong>blocks that satisfy the Kotlin compiler,by any method invocation the running application will cause a crash.</p><p>Since Kotlin also supports default method feature, they do not need to be overridden by the implementing classes since they provide default implementations. The contract remains fulfilled in this scenario.</p><p>Let’s see another example of the <strong>Shape </strong>implementation:</p><pre>class Rectangle(private val width: Double, private val height: Double) : Shape {<br>    override val name: String = &quot;Rectangle&quot;<br><br>    override fun area(): Double {<br>        return width * height<br>    }<br><br>    override fun perimeter(): Double {<br>        return 2 * (width + height)<br>    }<br>}</pre><p>The new class also satisfies the <strong>Shape </strong>interface contract same as the <strong>Circle </strong>class before. With both classes adhering to the same contract we are able see the potential benefits of programming to an interface, such as encapsulation, decoupling and other ones mentioned in the introduction segment.</p><h4>Type Checking</h4><p>In OOP, the concept of polymorphism allows objects of different classes to be treated as objects of a common superclass or interface. This feature enables us to write more flexible and reusable code by programming to an interface.</p><p>To use polymorphism in Kotlin, we can create a function that accepts an interface as a parameter in its signature. This can enable a function to be used with any object that implements that interface. We’ll continue with our <strong>Shape </strong>example from previous sections:</p><pre>fun someFunctionName(shape: Shape) {<br>   print(shape.name) // Prints the name of a concrete Shape object to the console<br>}</pre><p>Since our function takes <strong>Shape</strong> as one of the parameters, we can use any object that implements that contract with it:</p><pre>fun main(){<br>  val circle = Circle(2.0)<br>  val rectangle = Rectangle(2.0,2.0)<br><br>  someFunctionName(circle) // Prints “Circle”<br>  someFunctionName(rectangle) // Prints “Rectangle”<br>}</pre><p>We can already see the potential of programming to an interface. What limits us to change the implementation details of the <strong>Circle </strong>and <strong>Rectangle </strong>objects and keep using the same functions and what keeps us from creating new objects that implement the <strong>Shape</strong> interface? The rest of our project’s code will remain unchanged.</p><p>We can utilize interfaces for <a href="https://en.wikipedia.org/wiki/Type_system">type checking</a> as well. At times we might need to check if an object implementing an interface is of a specific class. Kotlin provides the <strong>is</strong> keyword to perform type checking. The <a href="https://kotlinlang.org/docs/typecasts.html">smart casting</a> feature casts the object to the target type within the scope of the type check.</p><p>We can create a function that type checks an object implementing the <strong>Shape</strong> interface and performs a custom action for <strong>Circle</strong> object instances:</p><pre>fun someDifferentFunction(shape: Shape) {<br>   if (shape is Circle) {<br>       val doubleTheRadius = shape.radius * 2<br>       println(&quot;The updated circle diameter is $doubleTheRadius&quot;)<br>   } else {<br>       println(&quot;This is not a circle!&quot;)<br>   }<br>}</pre><p>In the above code, if the type check is successful, Kotlin’s smart casting feature automatically casts the <strong>shape</strong> object to a <strong>Circle</strong> instance, allowing you to access its <strong>radius</strong> property. Now, let’s update our <strong>main </strong>function and see how the new function can be applied:</p><pre>fun main() {<br>   val circle = Circle(5.0)<br>   val rectangle = Rectangle(4.0, 6.0)<br><br>   someDifferentFunction(circle) // Prints: The updated circle diameter is 10.0<br>   someDifferentFunction(rectangle) // prints: This is not a circle!<br>}</pre><p>Even at this point we can see some of the potential benefits of programming to an interface with a versatile way of implementing different project strategies.</p><h4>Interface Inheritance and Composition</h4><p>Interface composition can be a powerful technique that enables us to write clean, modular, and flexible code. With this we can encourage the <em>interface segregation </em>and <em>separation of concerns</em> core principles of <a href="https://en.wikipedia.org/wiki/SOLID">SOLID</a> design guidelines. It should be noted that by composing small, focused interfaces our classes should only contain the implementation details it needs.</p><p>In Kotlin, we we have to separate the interface names with a comma (<strong>,</strong>) after the colon (<strong>:</strong>) in order to inherit from multiple interfaces:</p><pre>interface MixedInterface : InterfaceA, InterfaceB {<br>    // Additional properties and methods<br>}</pre><p>In order to extends the functionality of our <strong>Shape </strong>objects, let’s create two additional interfaces, <strong>Drawable</strong> and Resizable, to work with the <strong>Shape</strong> interface from the previous sections:</p><pre>interface Drawable {<br>    fun draw()<br>}<br><br>interface Resizable {<br>    fun resize(factor: Double)<br>}</pre><p>With the two new interfaces in place, we can now create a new contract for an <strong>AdvancedShape</strong> that will have the functionality and properties from them:</p><pre>interface AdvancedShape : Shape, Drawable, Resizable // no need for the body curly braces</pre><p>We can now create new <em>advanced </em>shapes:</p><pre>class AdvancedCircle(val radius: Double) : AdvancedShape {<br>    override val name: String = &quot;AdvancedCircle&quot;<br><br>    override fun area(): Double {<br>        return Math.PI * radius * radius //<br>    }<br><br>    override fun perimeter(): Double {<br>        return 2 * Math.PI * radius<br>    }<br><br>    override fun draw() {<br>        println(&quot;Drawing a $name with radius $radius.&quot;)<br>    }<br><br>    override fun resize(factor: Double) {<br>        val newRadius = radius * factor<br>        println(&quot;Resizing the $name to new radius of $newRadius.&quot;)<br>    }<br>}</pre><p>As we can see the new <strong>AdvancedCircle</strong> class provides implementations for the <strong>name</strong> property and the <strong>area</strong>, <strong>perimeter</strong>, <strong>draw</strong>, and <strong>resize</strong> methods.</p><h4>Dependency Injection and Inversion of Control</h4><p>In order to further solidify programming to an interface can improve our overall project structure and dependency chains, we should look at how interfaces play an important role in implementing these principles.</p><p><strong>Dependency Injection</strong></p><p><a href="https://en.wikipedia.org/wiki/Dependency_injection">Dependency Injection</a> (DI) is a design pattern that enforces the method of decoupling multiple components in a software system. In DI, dependencies between components are <em>provided</em> (or “<em>injected</em>”) externally rather than being hard-coded. With this method, we can make components more modular, testable, and maintainable.</p><p>Interfaces are crucial in implementing DI. If we would define one interface for a dependency, we can swap out different implementations of the dependency without ever modifying the dependent component.</p><p><strong>Inversion of Control</strong></p><p><a href="https://en.wikipedia.org/wiki/Inversion_of_control">Inversion of Control </a>(IoC) is a design principle that delegates the control of dependency management from individual components to an external container or framework. IoC enforces decoupling and flexibility by inverting the traditional flow of control, where components create and manage their dependencies. Using interfaces in combination with the IoC principle enables you to create modular, and maintainable solutions.</p><p>In order to demonstrate the true power of DI and IoC let’s see the following example of a proposed payment processor, and don’t worry about the somewhat increased lines of code than in the previous examples, we are progressing aren’t we?</p><p>Let’s start with defining the <em>payment processor </em>interface contract:</p><pre>interface PaymentProcessor {<br>   fun processPaymentTransaction(amount: Double, currency: String): Boolean<br>}</pre><p>When we have defined the general contract, now we can define a couple of example payment processors:</p><pre>class StripeProcessor : PaymentProcessor {<br>   override fun processPaymentTransaction(amount: Double, currency: String): Boolean {<br>       println(&quot;Stripe processing: $amount $currency&quot;)<br>       return true<br>   }<br>}<br><br><br>class PayPalProcessor : PaymentProcessor {<br>   override fun processPaymentTransaction(amount: Double, currency: String): Boolean {<br>       println(&quot;PayPal processing: $amount $currency&quot;)<br>       return true<br>   }<br>}</pre><p>Let’s now create an imaginary online store component that depends on the <em>PaymentProcessor</em> contract rather than some concrete payment processor implementation:</p><pre>class SomeInterestingShop(private val paymentProcessor: PaymentProcessor) {<br>   fun checkoutCart(amount: Double, currency: String): Boolean {<br>       val transactionResult = paymentProcessor.processPaymentTransaction(amount, currency)<br>       if (transactionResult) {<br>           println(&quot;Payment successful! Thank you for your purchase.&quot;)<br>       } else {<br>           println(&quot;Payment failed! Please try again.&quot;)<br>       }<br>       return transactionResult;<br>   }<br>}</pre><p>Our <em>main </em>function should look something like this now:</p><pre>fun main() {<br>   val stripeProcessor = StripeProcessor()<br>   val payPalProcessor = PayPalProcessor()<br><br><br>   val storeWithStripe = SomeInterestingShop(stripeProcessor)<br>   val storeWithPayPal = SomeInterestingShop(payPalProcessor)<br><br><br>   storeWithStripe.checkoutCart(10.0, &quot;EUR&quot;) // Prints: Stripe processing: 10.0 EUR<br>   storeWithPayPal.checkoutCart(7500.0, &quot;RSD&quot;) // Prints: PayPal processing: 7500.0 RSD<br>}</pre><p>If we follow the DI principle, we can <em>inject </em>the required <em>PaymentProcessor</em> implementations (StripeProcessor and PayPalProcessor) at <a href="https://www.techtarget.com/searchsoftwarequality/definition/runtime">runtime</a> which inevitably makes the <em>SomeInterestingShop </em>component more modular and adaptable to different situations.</p><h4>Testing and Mocking</h4><p>When we want to write new <a href="https://www.techtarget.com/searchsoftwarequality/definition/unit-testing">unit tests</a>, we aim to isolate the testing unit of code from any dependencies it might have. This can be frustrating at times, especially in legacy code blocks, with some components (units in this case) being dependant on other concrete components. Whatever test we write for a component that depends on some other concrete implementation details won’t be a unit test by definition, it will be an integrated test between two or more components. One possible way to achieve the necessary isolation between two testing components is by using test doubles, such as mocks or stubs. Mocks and stubs replace the actual dependencies during testing. When we use interfaces we make it easy on ourselves to create test doubles since they define the contract that the test double must <em>stick</em> to.</p><p>There are several testing frameworks available for Kotlin that can help us create test doubles, such as <a href="https://site.mockito.org/">Mockito</a> or <a href="https://mockk.io/">MockK</a>. Any mocking framework should allow you to create mocks and stubs for interfaces and even provide some kind of functionality for verifying interactions between your code and its dependencies.</p><p>We’ll continue with our fictional <em>PaymentProcessor</em> solution and provide the next unit test example for our <em>SomeInterestingShop </em>using the MockK framework.</p><p>Let’s first add the <a href="https://junit.org/junit5/">JUnit5</a> and MockK frameworks to our project as a <a href="https://gradle.org/">Gradle</a> dependency and yes, I do realize that I’ve never specified that this example project uses the Gradle build system, but detailing such steps would definitely fall out of the scope for this, already long-ish, article topic. All the source code for the examples given in this article are available on this <a href="https://github.com/Remthix/programming-to-interface-kotlin-examples">GitHub</a> repository.</p><p>To add the MockK dependency, we’ll put the following in our<em> build.gradle.kst </em>file:</p><pre>dependencies {<br>   // Other dependencies<br>   // JUnit5<br>   testImplementation(&quot;org.junit.jupiter:junit-jupiter-api:5.9.2&quot;) //Check for latest versions<br>   testRuntimeOnly(&quot;org.junit.jupiter:junit-jupiter-engine:5.9.2&quot;) //Check for latest versions<br><br><br>   // MockK<br>   testImplementation(&quot;io.mockk:mockk:1.13.5&quot;) //Check for latest versions<br>}</pre><p>We’ll next sync our project and proceed with creating the <em>SomeInterestingShop </em>unit test class<em> </em>(<em>test </em>project package):</p><pre>import io.mockk.every<br>import io.mockk.mockk<br>import io.mockk.verify<br>import org.junit.jupiter.api.Assertions.assertTrue<br>import org.junit.jupiter.api.Test<br><br>class SomeInterestingShopTest {<br>   @Test<br>   fun `process payment and return true`() {<br>       // Arrange<br>       val mockPaymentProcessor = mockk&lt;PaymentProcessor&gt;()<br>       every { mockPaymentProcessor.processPaymentTransaction(any(), any()) } returns true<br><br>       val onlineStore = SomeInterestingShop(mockPaymentProcessor)<br><br>       // Act<br>       val result = onlineStore.checkoutCart(100.0, &quot;RSD&quot;)<br><br>       // Assert<br>       verify { mockPaymentProcessor.processPaymentTransaction(100.0, &quot;RSD&quot;) }<br>       assertTrue(result, &quot;Payment was successful.&quot;)<br>   }<br>}</pre><p>We can see how easy it is for the MockK framework to create a mock <strong>PaymentProcessor</strong>. We configure/drive the mock to return <strong>true</strong> whenever its <em>processPaymentTransaction</em> method is called. We then inject the mock within a new <em>SomeInterestingShop</em> instance and call its <em>checkoutCart</em> method. In the final step, we verify that the <em>processPaymentTransaction</em> method was indeed called with the expected arguments.</p><p>Interfaces combined with powerful mocking frameworks like MockK, can deliver more focused and maintainable unit tests for our projects.</p><h4>Use Cases</h4><p>So, we slowly approaching the end of our little journey with exploring interfaces and contracts, but we are not finished yet. After laying some ground-work in the previous sections, we can now look at some examples with different design patterns.</p><p><strong>Adapter Pattern</strong></p><p>The <a href="https://en.wikipedia.org/wiki/Adapter_pattern">Adapter</a> pattern is a structural design pattern that allows incompatible interfaces to work together. With creating an adapter interface, we can invoke a bridge between classes with incompatible interfaces. This makes it easier to scale our application or integrate with new third-party solutions.</p><p>An example would be to integrate our project with different analytics services, such as Google Analytics and Flurry. The new <em>AnalyticsAdapter</em> interface and its implementations can be something like this:</p><pre>interface AnalyticsAdapter {<br>   fun trackEvent(eventName: String, properties: Map&lt;String, Any&gt;)<br>}<br><br>class GoogleAnalyticsAdapter : AnalyticsAdapter {<br>   // Implementation details...<br>   override fun trackEvent(eventName: String, properties: Map&lt;String, Any&gt;) {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}<br><br>class FlurryAdapter : AnalyticsAdapter {<br>   // Implementation details...<br>   override fun trackEvent(eventName: String, properties: Map&lt;String, Any&gt;) {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}</pre><p>Now we can add different analytics services to our project much more effectively.</p><p><strong>Strategy Pattern</strong></p><p>The <a href="https://en.wikipedia.org/wiki/Strategy_pattern">Strategy</a> pattern is a behavioral design pattern that enables us to define a family of algorithms, encapsulate them, and make them interchangeable. With the strategy pattern interface we can easily switch between different algorithms at runtime without modifying the code that uses the algorithms.</p><p>Let’s explore a file compression utility that supports different compression algorithms, such as ZIP, RAR, and 7z. You could create a <em>CompressionStrategy</em> interface and specific implementations for each compression algorithm:</p><pre>package patterns<br><br>import java.io.File<br><br><br>interface CompressionStrategy {<br>   fun compress(inputFiles: List&lt;File&gt;, outputFile: File)<br>}<br><br><br>class ZipCompressionStrategy : CompressionStrategy {<br>   override fun compress(inputFiles: List&lt;File&gt;, outputFile: File) {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}<br><br><br>class RarCompressionStrategy : CompressionStrategy {<br>   override fun compress(inputFiles: List&lt;File&gt;, outputFile: File) {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}<br><br><br>class SevenZipCompressionStrategy : CompressionStrategy {<br>   override fun compress(inputFiles: List&lt;File&gt;, outputFile: File) {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}</pre><p><strong>Repository Pattern</strong></p><p>A widely used design pattern is the <a href="https://cubettech.com/resources/blog/introduction-to-repository-design-pattern/">repository</a> pattern. With the repository pattern we can abstract abstract any data access logic for a project. With the proposed repository interface we can seemingly switch between different data sources (<a href="https://docs.oracle.com/javaee/6/tutorial/doc/gijqy.html">REST</a> APIs, databases, or in-memory caches) without modifying the code that depends on the repository.</p><p>We can play around and create a dummy task management solution that would manage our tasks and support different storage systems, such as local databases, cloud databases, or in-memory storage.</p><p>Let’s create a <strong>TaskRepository</strong> interface and the various implementations for different storage systems:</p><pre>package patterns<br><br>data class Task(val id: String, val title: String, val description: String, val completed: Boolean)<br><br>interface TaskRepository {<br>   fun getTask(id: String): Task?<br>   fun addTask(task: Task): Task<br>   fun updateTask(task: Task): Task<br>   fun deleteTask(id: String): Boolean<br>   fun getAllTasks(): List&lt;Task&gt;<br>}<br><br>lass InMemoryTaskRepository : TaskRepository {<br>   // Implementation details...<br>   override fun getTask(id: String): Task? {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun addTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun updateTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun deleteTask(id: String): Boolean {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun getAllTasks(): List&lt;Task&gt; {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}<br><br><br>class LocalDatabaseTaskRepository : TaskRepository {<br>   // Implementation details...<br>   override fun getTask(id: String): Task? {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun addTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun updateTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun deleteTask(id: String): Boolean {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun getAllTasks(): List&lt;Task&gt; {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}<br><br><br>class CloudDatabaseTaskRepository : TaskRepository {<br>   // Implementation details...<br>   override fun getTask(id: String): Task? {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun addTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun updateTask(task: Task): Task {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun deleteTask(id: String): Boolean {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br><br>   override fun getAllTasks(): List&lt;Task&gt; {<br>       TODO(&quot;Not yet implemented&quot;)<br>   }<br>}</pre><p>When we utilize our <em>TaskRepository </em>interface other parts of the task management solution remain agnostic to storage system implementation detail that’s going to being used. As with the other examples in this sections, this method ensures more scalability, maintainability, and the ability to switch between different data sources without modifying the code that depends on the repository.</p><p>These just some use cases that showcase the power of programming to an interface. By going this route in our software solutions, we can that much make our lives as software developers a tad easier to scale and adapt to frequent change requirements.</p><h3>Final Note</h3><p>In this article, I tried to introduce to you dear Reader the concept of programming to an interface. With this technique, we all as developers can create software solutions that gain all the befits already mentioned throughout the article.</p><p>With adopting programming to an interface in our projects, we cultivate a new design mindset that prioritizes extensibility and modularity. This approach sets the stage for building solutions capable of adapting to new challenges and opportunities, while minimizing the potential for technical debt and codebase stagnation. As we continue working on software projects, we should remember the importance of programming to an interface and apply this technique to design software that is both resilient and flexible, making it easier to scale, modify, and maintain throughout its lifecycle.</p><p>I hope that you found this article informative and insightful. Your feedback is important in improving this and any future content. I aim to provide you with the best resources possible. Feel free to share your thoughts, suggestions, or any questions you may have. As mentioned earlier, all the source code examples are available <a href="https://github.com/Remthix/programming-to-interface-kotlin-examples">here</a>.</p><p>Thank you for reading, and I look forward in hearing your thoughts about the subject matter. Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ad65d437f97" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Branchless programming — Why your CPU will thank you]]></title>
            <link>https://sdremthix.medium.com/branchless-programming-why-your-cpu-will-thank-you-5f405d97b0c8?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/5f405d97b0c8</guid>
            <category><![CDATA[performance-optimization]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[developer-growth]]></category>
            <category><![CDATA[branchless-programming]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Sun, 09 Apr 2023 20:24:12 GMT</pubDate>
            <atom:updated>2024-06-01T11:50:37.395Z</atom:updated>
            <content:encoded><![CDATA[<h3>Branchless programming — Why your CPU will thank you</h3><figure><img alt="Old tree branches — branchless programming series" src="https://cdn-images-1.medium.com/max/1024/1*vSMC6FIa5aVkIBji8um0wQ.png" /></figure><h3><strong>Intro</strong></h3><p>‘Performance’… we hear it everywhere in software development nowadays. But, what does that mean? We all can have different definitions for ‘performance’ in our daily workflows. On a very lower-level, performance is associated with the underlying computer components that we utilize.</p><p>Performance optimization plays a crucial role in extracting maximum efficiency from contemporary processors (CPUs). An often overlooked technique that might be holding the key to substantial performance gains, is branchless programming. In this article we aim to provide a comprehensive understanding of branchless programming, exploring the techniques utilized, its practical applications, drawbacks, and the impact it has on contemporary CPUs.</p><p>Branchless programming is a programming approach that, as its name suggests, seeks to minimize or eliminate branches, such as conditional statements (<strong>if-else</strong> statements) and <em>loops</em>, which can introduce performance penalties due to pipeline slowdowns and wrongfully predicted branches in modern processors. As CPU architectures have advanced, the depth of their instruction pipelines has increased, making the cost of mispredicted branches even more impactful. These penalties occur when a processor incorrectly predicts the outcome of a branch, causing it to discard the work already done and start again, wasting valuable processing cycles.The primary idea behind branchless programming is to mitigate these performance penalties by replacing branches with more efficient arithmetic and bitwise operations, which can be executed very efficiently on modern processors. With the reduction of the number of branches, this technique can lead to significant performance improvements in performance-critical code, such as high-performance computing, real-time systems, or graphics and <a href="https://en.wikipedia.org/wiki/Ray_tracing_(graphics)">ray-tracing</a> processing.</p><p>There are several techniques used in branchless programming, including replacing conditional assignments with bitwise operations, using conditional move instructions, employing lookup tables, and utilizing arithmetic operations. Each technique offers unique advantages that can be used depending on the specific requirements and characteristics of the code in need of optimization. With understanding and utilization of these techniques, software developers can extract that much more performance from their code without compromising its functionality. It is important to note that while branchless programming can produce substantial performance benefits, it may also introduce some drawbacks. One such drawback is the potential for reduced code readability and maintainability. This can lead to situations where its difficult for other programmers to understand, debug, and change the existing code. Additionally, some branchless programming techniques can introduce security risks, such as cache timing attacks. As a result, it is essential for programmers to balance the performance gains against the potential drawbacks before implementing branchless programming techniques.</p><p>In this article, we will explore the various techniques used in branchless programming, providing examples in the C programming language to illustrate each concept. We will also mention some practical applications of branchless programming, focusing on the types of systems and applications where it is most beneficial to be implemented. Finally, we will examine the drawbacks and limitations of branchless programming, offering some insights on when to apply these techniques and when not to. A lot of the optimization techniques we are going to mention are mostly covered in CPU manufacture manuals.</p><p>The C programming language has been chosen for the given examples as it is more closely associated with performance-critical systems and low-level programming. Using C for the examples will provide a clearer illustration of the performance benefits and implications of branchless programming.</p><p>At the end of the article, we should be a bit more knowledgeable about this subject matter in order to make informed decisions when and where to incorporate branchless programming techniques into our own code, aiming for the perfect balance between performance optimization and code readability and maintainability.</p><p>Let’s try and unlock the full performance potential of modern CPUs.</p><h3>Branchless techniques</h3><h4>Conditional Assignments vs. Bitwise Operations</h4><p>One of the most frequently-used techniques in branchless programming is replacing conditional assignments with <a href="https://en.wikipedia.org/wiki/Bitwise_operation">bitwise operations</a>.</p><p>Bitwise operations directly manipulate the bits that represent the data in a computer’s memory, making them very efficient and suitable for performance-critical code. The most common bitwise operations used in branchless programming are AND, OR, and XOR.</p><p>AND, OR, and XOR operations work on the individual bits of the operands, producing result values based on the corresponding bits of the input values. The AND operation results in a 1 for each bit position where both input bits are 1, while the OR operation results in a 1 for each bit position where either input bit is 1.</p><p>In branchless programming bitwise operations can replace conditional assignments in order to eliminate branches and improve performance. We can look at the following example for a conditional assignment:</p><pre>int a = 5;<br>int b = 10;<br>int max;<br><br>if (a &gt; b) {<br>    max = a;<br>} else {<br>    max = b;<br>}</pre><p>In the example code above, we use an <em>if-else </em>statement to determine the maximum of two integer values, <strong>a</strong> and <strong>b</strong>. This introduces an unnecessary branch that may lead to performance penalties on some CPU’s. Using bitwise operations we can achieve the same result without introducing any branches:</p><pre>int a = 5;<br>int b = 10;<br>int max;<br><br>int greater = a - ((a - b) &amp; ((a - b) &gt;&gt; (sizeof(int) * CHAR_BIT - 1)));</pre><p>In this branchless version, we first calculate the difference between <strong>a</strong> and <strong>b</strong>, then create a mask by shifting the sign bit of the difference to the right by the number of bits in an integer minus 1. The mask will have all bits set to 1 if the difference is negative (i.e., <strong>a &lt; b</strong>), and all bits set to 0 if the difference is positive (i.e., <strong>a &gt;= b</strong>). We then AND the mask with the difference and subtract the result from <strong>a</strong> to obtain the greater value.</p><p>This approach is particularly useful when the performance cost of branches is significant, such as in <a href="https://en.wiktionary.org/wiki/tight_loop">tight loops</a> or performance-critical code. Bitwise operations are typically faster than branching instructions, as they do not introduce <a href="https://en.wikipedia.org/wiki/Pipeline_stall">pipeline stalls</a> or mispredicted branches, allowing the processor to execute the code more efficiently with not requiring additional computing cycles.</p><p>As we mentioned earlier, replacing conditional assignments with bitwise operations can sometimes result in less readable and maintainable code. The branchless version of the code may be more difficult to understand at first glance, making it harder for other developers to debug or modify the code. As such, it is essential to make clear decisions when and where to pick performance gains over code readability, and where to skip branchless programming techniques altogether.</p><p>In addition to the AND, OR, and XOR operations, other bitwise operations, such as NOT and bit shifts, can also be used in branchless programming to achieve various results. The key is to understand the underlying principles of these operations and how they can be used to eliminate or minimize branches in our code.</p><h4>Conditional Move Instructions</h4><p>Conditional <em>move</em> instructions are another powerful technique used in branchless programming. These instructions allow data to be moved conditionally without creating a branch, which can lead to significant performance improvements in modern processors.</p><p>While some assembly languages, such as <a href="https://en.wikipedia.org/wiki/X86_assembly_language">x86 assembly</a>, provide specific conditional move instructions like CMOV, the C programming language does not have a direct equivalent. However, we can mimic the behavior of conditional move instructions using a combination of bitwise operations and arithmetic.</p><p>Consider the following example, where we again find the maximum of two integer values <strong>a</strong> and <strong>b</strong>:</p><pre>int a = 5;<br>int b = 10;<br>int max;<br><br>if (a &gt; b) {<br>    max = a;<br>} else {<br>    max = b;<br>}</pre><p>We can replace the if-else statement with a branchless equivalent that simulates a conditional move:</p><pre>int a = 5;<br>int b = 10;<br>int max;<br><br>int mask = (a &gt; b) - 1; <br>max = (a &amp; mask) | (b &amp; ~mask);</pre><p>In this example, we first create a mask based on the result of the comparison <strong>(a &gt; b)</strong>. If <strong>a</strong> is greater than <strong>b</strong>, the mask will be all 1s (0x00000000), and if <strong>a</strong> is less than or equal to <strong>b</strong>, the mask will be all 0s (0xFFFFFFFF). We then AND the mask with <strong>a</strong> and the negation of the mask with <strong>b</strong>, combining the results with an OR operation to obtain the maximum value.</p><p>This branchless version avoids the performance penalties associated with branching, allowing the processor to execute the code more efficiently. As with other branchless programming techniques, it can also result in less readable and maintainable code. Can we spot a pattern here? It is essential to weigh the performance benefits against these potential drawbacks and apply the technique carefully.</p><p>At this point, it’s worth mentioning that some compilers, especially modern ones, may automatically optimize certain types of conditional assignments into conditional move instructions when targeting specific processor architectures. However, relying on compiler optimizations can be unpredictable, and explicitly implementing branchless programming techniques can help ensure consistent performance gains. In cases where the C programming language is compiled to assembly for a specific target architecture that supports conditional move instructions, the compiler may generate more efficient machine code by automatically replacing certain branches with conditional moves. This optimization is not guaranteed, and writing branchless code in C can help ensure consistent performance improvements regardless of the target architecture or compiler optimizations.</p><h4>Lookup Tables</h4><p>A lookup table is a precomputed array of values that can be indexed directly to retrieve a result, avoiding the need for any runtime execution of complex calculations or conditional logic.</p><p>Lookup tables are particularly useful for replacing <em>switch</em> statements or series of <em>if-else</em> statements, which can introduce multiple branches and negatively impact performance on modern CPUs. The replacing of these branches with direct array indexing, lookup tables can significantly speed up performance-critical code.</p><p>We can look at a simple example that calculates the factorial of a small integer value, <strong>n</strong>:</p><pre>int n = 5;<br>int factorial;<br><br>switch (n) {<br>    case 0:<br>        factorial = 1;<br>        break;<br>    case 1:<br>        factorial = 1;<br>        break;<br>    case 2:<br>        factorial = 2;<br>        break;<br>    case 3:<br>        factorial = 6;<br>        break;<br>    case 4:<br>        factorial = 24;<br>        break;<br>    case 5:<br>        factorial = 120;<br>        break;<br>    default:<br>        factorial = -1; // Invalid input<br>}</pre><p>The above code uses a switch statement to calculate the factorial, introducing multiple branches. We can replace the switch statement with a branchless lookup table:</p><pre>int n = 5;<br>int factorial;<br><br>const int lookupTable[] = {1, 1, 2, 6, 24, 120};<br>factorial = (n &lt; sizeof(lookup_table) / sizeof(lookup_table[0])) ? lookupTable[n] : -1;</pre><p>In this example, we first create a lookup table as a constant array containing precomputed factorial values. We then directly index the lookup table using the input value <strong>n</strong> and assign the result to <strong>factorial</strong>. To handle invalid input, we use the <em>ternary</em> conditional operator to check if <strong>n</strong> is within the bounds of the lookup table before indexing it. If the input is out of bounds, the result is set to -1.</p><p>The branchless version eliminates the multiple branches introduced by the switch statement, allowing the processor to execute the code more efficiently. It is important to note that lookup tables trade memory for performance, as the precomputed values must be stored in memory. In some cases, this compromise may be the preferred way, particularly when memory space is not a limiting factor and the performance gains are significant.</p><p>Lookup tables can be used in a wide range of scenarios, from simple value mappings to more complex functions. They are particularly useful for replacing branches in performance-critical code, such as in tight loops, real-time systems, or graphics processing. With this technique, its important to take memory usage into account when going with the implementation route.</p><h4>Arithmetic Operations</h4><p>Arithmetic operations provide another powerful tool in the branchless programming world. With the efficiency of arithmetic operations, such as addition, subtraction, multiplication, and division, we can write code solutions that not only circumvent the pitfalls of branching but also amazes with its performance. The task of utilizing arithmetic operations in branchless programming is based upon the idea that we can replace conditional logic with mathematical expressions. These expressions are often quite ingenious, leaving even the most jaded of programmers with a sense of wonder and approval for their elegance.</p><p>We can look at the following example where we have to determine the absolute value of an integer <strong>x</strong>:</p><pre>int x = -42; // Little reference<br>int absX;<br><br>if (x &gt;= 0) {<br>    absX = x;<br>} else {<br>    absX = -x;<br>}</pre><p>The code above, while functional, suffers from the branching issue we try to omit. We will now try to remove the branch by invoking the powers of arithmetic operations:</p><pre>int x = -42;<br>int absX;<br><br>int mask = x &gt;&gt; (sizeof(int) * CHAR_BIT - 1);<br>absX = (x + mask) ^ mask;</pre><p>In this branchless code block, we create a mask by shifting the sign bit of <strong>x</strong> to the right by the number of bits in an integer minus one. The mask will be a row of 1s (0xFFFFFFFF) if <strong>x</strong> is negative, and a row of 0s (0x00000000) if <strong>x</strong> is positive or zero. We then add the mask to <strong>x</strong> and XOR the result with the mask, producing the absolute value of <strong>x</strong>.</p><p>This transformation produces code that not only avoids any branching penalties it could suffer from, but also provides some, if we dare to say, mathematical elegance.</p><p>Arithmetic operations can be used in a variety of scenarios, limited only by the bounds of ones imagination and mathematical abilities.The author of this article is definitely not akin to this subject matter. These operations provide an alternative to branching, enabling our code to support a processor’s pipeline is a more native approach.</p><p>The branchless programming techniques we have examined, including replacing conditional assignments with bitwise operations, conditional move instructions, lookup tables, and arithmetic operations, can elevate our code to new heights of performance. When used carefully, these techniques can help us tackle the full potential of modern processors while adding a touch of coding <em>extravaganza</em>. These techniques elevate our programming skills to the next level.</p><p>As we have seen, branchless programming is much like an art form, blending mathematics, logic, and creativity to overcome the limitations of branching.</p><h3>Practical applications</h3><p>Branchless programming techniques can be applied to a wide range of applications, particularly those where performance is critical. In this section, we will explore several practical applications of branchless programming, examining the benefits and challenges introduced with each use-case. While the previous examples were intentionally designed to be more introductory, the following will be more focused on providing concrete, real-world examples of how branchless programming can be employed.</p><h4>Performance-critical systems</h4><p>In performance-critical systems, such as real-time systems, and high-performance computing applications, branchless programming can provide significant performance gains by minimizing the overhead associated with branching. Examples of performance-critical systems include automotive control systems, real-time gaming, medical devices, and high-frequency trading platforms.</p><p>In these systems, minimizing latency and maximizing throughput are often top priorities, and branchless programming techniques can help achieve these goals in ways we mentioned before. Using techniques such as conditional move instructions, lookup tables, and arithmetic operations, developers can create code that is more efficient and better suited for high-performance environments.</p><h4>Graphics processing</h4><p>Graphics processing is another area where branchless programming can provide substantial benefits. Modern graphics processing units (GPUs) are designed to handle large amounts of parallelism and are particularly sensitive to branching instructions. With the reduction of branches in graphics processing code, developers can take full advantage of the GPU’s parallel processing capabilities, resulting in improved performance and reduced latency.</p><p>Examples of graphics processing applications where branchless programming can be applied include shader programs,geometry processing, texture filtering, procedural environment generation and even <a href="https://www.weka.io/learn/hpc/gpus-for-machine-learning/">AI/ML</a> model trainings. In these applications, employing techniques such as bitwise operations and lookup tables can help minimize branching and improve overall performance.</p><h4>Cryptographic algorithms</h4><p>Always a popular topic, cryptographic algorithms are providing another practical application of branchless programming techniques. Many cryptographic algorithms require constant-time execution to prevent <a href="https://en.wikipedia.org/wiki/Timing_attack">timing attacks</a>, which can leak sensitive information based on variations in execution time. At the time of writing this article (April of 2023), there’s an influx of data breaching and scrapping of all types of sensitive information throughout the world in with the sole purpose to wait for when the time comes and computers get powerful enough to crack the encryptions used on that data and be utilized in potential malicious ways. Branchless programming can help ensure constant-time execution by eliminating branches, which may introduce variable execution times.</p><p>The branchless programming techniques in cryptographic algorithms, can be used by developers to create a tad-bit more secure implementations that are less susceptible to timing attacks. Some of the examples of cryptographic algorithms that can benefit from branchless programming include hash functions, <a href="https://en.wikipedia.org/wiki/Block_cipher">block ciphers</a>, and public-key cryptography algorithms.</p><h4>Compression algorithms</h4><p>Compression algorithms are another area where branchless programming can provide performance improvements. Compression algorithms usually involve complex operations and decision-making processes, which can lead to numerous branches and potential performance bottlenecks. With branchless programming, we can create more efficient compression algorithms that could yield faster compression and decompression times.</p><p>Compression algorithms where branchless programming can be applied include data compression algorithms like <a href="https://en.wikipedia.org/wiki/LZ77_and_LZ78">LZ77</a>, <a href="https://en.wikipedia.org/wiki/LZ77_and_LZ78">LZ78</a>, and <a href="https://en.wikipedia.org/wiki/Huffman_coding">Huffman</a> coding, as well as image compression algorithms like JPEG and PNG. In these applications, techniques such as conditional move instructions, lookup tables, and arithmetic operations can help reduce branching and improve overall performance.</p><h4>Networking and communication protocols</h4><p>Networking and communication protocols often require high-performance and low-latency throughout to ensure smooth and efficient data transfer between various devices. The implementation of optimized protocols that use branchless programming in order to minimize the overhead associated with branching can lead to overall performance gains.</p><p>Networking transport layer protocols like TCP and UDP are just some of the examples where branchless programming can help. In these applications, using techniques such as bitwise operations, lookup tables, and arithmetic operations can help reduce branching and improve the efficiency of protocol processing.</p><h4>Database Systems</h4><p>High-performance database systems like <a href="https://sqlite.org/index.html">SQLite</a>, utilize branchless programming techniques to optimize query processing, sorting, and indexing operations. These techniques help reduce pipeline stalls and cache misses, enabling faster access and retrieval of data.</p><h4>Embedded Systems</h4><p>Branchless programming is often employed in the development of embedded systems, where power consumption and resource constraints are critical factors. With the reduction of the number of branch instructions, developers can create more efficient code that reduces power consumption and improves battery life in devices like smartphones and IoT sensors.</p><p>There is a broad range of practical applications, particularly in performance-critical environments where branchless programming can help mitigate some of the pitfalls. Now developers can create more efficient code that takes full advantage of modern processor architectures and delivers improved performance in various applications. Like everything, it is essential to balance the performance gains with the potential drawbacks. When applied appropriately, branchless programming can be a powerful tool for optimizing code and unlocking the full potential of contemporary CPUs.</p><h3>Drawbacks and Limitations</h3><p>While we covered some of the performance benefits of what branchless programming could offer, it is worth noting that this comes with a set of potential drawbacks and limitations. In this section, we will explore some of the potential issues that can come when using branchless programming techniques, as well as situations where these techniques may not be the optimal choice.</p><h4>Code readability and maintainability</h4><p>Ok, we already mentioned this one a few times though this article, but the simple case it that this is indeed one of the most notable drawbacks of branchless programming, because it can lead to code that is more difficult to read and maintain. Techniques such as bitwise operations, lookup tables, and arithmetic operations can create code that is less intuitive and harder to understand compared to traditional branching code. This can make the code more challenging to debug, modify, or extend, potentially leading to a higher risk of introducing bugs or making it harder for other developers to work with the code.</p><p>When using branchless programming techniques, it is essential to balance the performance gains with the potential impact on code readability and maintainability. In some cases, it may be more appropriate to use traditional branching code for the sake of clarity, particularly when the performance gains are not significant or the code is not performance-critical or when developing in a more heterogenous developer teams.</p><h4>Compiler optimizations</h4><p>Modern compilers are more than capable of applying various optimizations to the generated code, including automatically converting some branches to branchless equivalents. Relying on compiler optimizations can sometimes result in more efficient code without the need for manual branchless programming. However, these optimizations are not always consistent or guaranteed, and explicitly implementing branchless techniques can help ensure consistent performance gains.</p><p>In some cases, branchless programming techniques may conflict with compiler optimizations or even result in less efficient code. It is essential to test and profile the code to ensure that the branchless techniques used are genuinely providing the expected performance improvements. Remember, when performance is involved, always measure. Don’t pre-optimize code before measuring it.</p><h4>Processor-specific performance characteristics</h4><p>This is mostly tailored to embedded programming, where each of the hardware piece requires some form of performance optimizations in order to squeeze that much more from what is possible. Branchless programming techniques are often designed to take advantage of specific performance characteristics of modern processors, such as pipelining and branch prediction that we mentioned. Not all processors have the same performance characteristics, and branchless programming techniques that work well on one processor may not be as effective on another and could be even made detrimental to the point of failure.</p><p>When using branchless programming techniques, it is essential to consider the target processor’s performance characteristics and ensure that the techniques used are appropriate for that processor. In some cases, it may be necessary to adapt or modify the branchless techniques to suit the specific performance characteristics of the target processor. Many processor manufactures provide manuals for each of their product, just for these kind of scenarios where each processor cycle matters. One example can be Intel’s architecture and optimization manual (<a href="https://cdrdv2-public.intel.com/671488/248966-046A-software-optimization-manual.pdf">PDF</a>).</p><h4>Trade-offs between performance and resource usage</h4><p>Some branchless programming techniques involve trade-offs between performance and resource usage, such as memory or computational complexity. For example, lookup tables can provide significant performance improvements by eliminating branches but may require additional memory to store the precomputed values. Similarly, arithmetic operations can reduce branching but may introduce additional computational complexity.</p><p>When using branchless programming techniques, it is essential to weigh the performance gains against the potential resource usage implications. In some cases, the trade-offs may not be worth the performance benefits, particularly when resources are limited or the performance gains are not substantial.</p><h4>Security Concerns</h4><p>While branchless programming techniques can offer significant performance improvements, they may also introduce security risks if not implemented correctly, particularly when used in cryptographic algorithms. One of the primary concerns is the potential for <a href="https://en.wikipedia.org/wiki/Side-channel_attack">side-channel</a> attacks. Side-channel attacks exploit information leaked through physical properties of a system, such as timing, power consumption, or electromagnetic radiation and focuses on specific ways in which algorithms are implemented. In cryptographic implementations, a constant-time execution is important for minimizing the risk of such attacks. Although some branchless techniques can help achieve constant-time execution, incorrect implementation can still lead to information leakage.</p><p>To address the security risks associated with branchless programming techniques, developers can execute the following:</p><ol><li>Thoroughly understand the potential side-channel vulnerabilities of their specific use-case and the branchless techniques they plan to use</li><li>Follow best practices and guidelines for implementing constant-time cryptographic algorithms, ensuring that execution time is independent of the input data</li><li>Test and validate the implementation for potential side-channel vulnerabilities using tools like static code analyzers, dynamic analysis, and <a href="https://en.wikipedia.org/wiki/Fuzzing">fuzz</a> testing</li><li>Keep up-to-date with the latest research, security vulnerabilities, and recommended mitigation strategies in the field.</li></ol><p>Just by being aware of the security concerns and mentioned best practices, developers can minimize the risks associated with branchless programming techniques while still benefiting from their performance improvements.</p><h4>Limited applicability</h4><p>While branchless programming techniques can be applied to a wide range of scenarios, there are some situations where these techniques may not be suitable or may provide limited benefits. For example, some algorithms or code structures may inherently require complex branching logic that cannot be easily replaced with branchless equivalents.</p><p>In such cases, it is essential to recognize the limitations of branchless programming and consider alternative optimization strategies, such as algorithmic improvements, parallelization, or hardware acceleration.</p><p>Branchless programming offers numerous performance benefits, it is essential to be aware of the potential drawbacks and limitations associated with these techniques. We should carefully consider the trade-offs and applicability of branchless programming. One way in which we as developers can make informed decisions, is to decide when and how to use these techniques to optimize our code effectively.</p><h3>Final Note</h3><p>Throughout this article, we have delved into the world of branchless programming, exploring its underlying principles, techniques, some practical applications, and potential drawbacks. Alongside branchless programming, it’s also important to consider other optimization techniques, such as <a href="https://specbranch.com/posts/intro-to-micro-optimization/">micro-optimizations</a>, which focus on making small, incremental improvements to the code’s performance.</p><p>Micro-optimizations can complement branchless programming techniques, helping developers to further refine their code and achieve even better performance gains. Combining branchless programming with micro-optimizations can enable developers to take a more holistic approach to code optimization, addressing various aspects of performance improvement.</p><p>At this point it is essential to look back on some of the key takeaways and consider how we, as developers, can apply this knowledge to our own projects, regular workflows and continue to explore the benefits of branchless programming. Branchless programming offers a powerful set of techniques that can help us unlock the full potential of modern processors, leading to significant performance improvements in various applications. From performance-critical systems and graphics processing to cryptographic algorithms and networking protocols, the practical applications of branchless programming are vast and varied. As with everything, it is crucial to recognize that this type of programming is not a <em>one-size-fits-all</em> solution. We have already seen that there are potential drawbacks and limitations to consider, such as the impact on code readability and maintainability, conflicts with compiler optimizations, processor-specific performance characteristics, and trade-offs between performance and resource usage. It is essential to approach branchless programming with a critical eye, carefully considering the specific context and requirements of each project to determine whether the benefits of these techniques outweigh the potential drawbacks.</p><p>As developers, our journey with branchless programming should not end with this article. There’s no shortage of processor architectures and optimization techniques to explore, with the whole landscape is constantly evolving, and it is essential for some developers to stay up-to-date with the latest developments in the field. Continuing to learn about new techniques, profiling tools, and best practices, we can ensure that we are well-equipped to utilize the power of branchless programming in our own projects. In addition, it is crucial to cultivate a culture of collaboration and knowledge-sharing within the developer community itself. Only by discussing our experiences with branchless programming, sharing insights, and providing constructive feedback, we can collectively see what is possible with these techniques and continue to drive innovation in the field.</p><p>If this article has sparked your interest in branchless programming and inspired you to delve deeper into the subject matter, feel free to leave a comment and feedback. With the insights gained from this exploration, you are now just a tad-bit better equipped to tackle performance challenges in your projects and bring forth optimized, efficient, and high-performing code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5f405d97b0c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Discovering: Quantum Computing]]></title>
            <link>https://sdremthix.medium.com/discovering-quantum-computing-bb8b4e53e74d?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb8b4e53e74d</guid>
            <category><![CDATA[qauntum-mechanics]]></category>
            <category><![CDATA[quantum-physics]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[quantum-computer]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 06 Feb 2023 06:39:19 GMT</pubDate>
            <atom:updated>2023-02-18T09:41:28.052Z</atom:updated>
            <content:encoded><![CDATA[<h4>Episode 1: The strange case of the Qubit</h4><figure><img alt="Discovering Quantum Computing series photo of the interference pattern using a particle beam light" src="https://cdn-images-1.medium.com/max/1024/1*D7ktT6UlWYV8JOdouRVhOA.jpeg" /></figure><h3>Intro</h3><p>The Discovering series is a new format I’m experimenting with, the idea for which came from the notion that some subjects require a bit more written content and, by extension, time to cover the basics over other similar articles.</p><p>For each Discovery subject at least two content ‘episodes’ are planned. The episodes are ordered in an incremental way and released in sequential order which is the same way they are meant to be read through.</p><p>This article episode also marks the first time I have to credit a co-author for it. In the world of machine learning and AI, there have been made significant strides in the past few years and one particular tool introduced itself to the world at the end of last year (2022) and that is <a href="https://openai.com/blog/chatgpt/"><strong>ChatGPT</strong></a>. It would be dishonest of me to not mention that I’ve used this tool during the writing of this article episode. In particular the helping hand of ChatGPT was needed for sorting and picking the right material from my previous notes in order to put more emphasis on terminology and questions that the readers would potentially find more interesting.</p><p>With the credits out of the way, we can start with the principal topic of this article.</p><p>The subject of <a href="https://en.wikipedia.org/wiki/Quantum_computing">quantum computing</a> is something I’ve been attempting to tackle for quite some time and tried to write about it for at least half an year now. One would be right in thinking that this isn’t a subject matter that is easy to explain let alone provide some validity and practical use cases for the implementation in this, relatively new industry. To say that what follows is a layman’s interpretation of a very complex subject matter is an understatement, but here goes nothing! Let’s dive into the amazing world of quantum computing.</p><h3>Basic concepts</h3><p>To cover the subject of the qubit (quantum bit) and quantum computing as a whole, we should take a few moments and look into quantum mechanics in search for answers.</p><p>It has always been fascinating to physicists to study the behavior of matter and energy data at an atomic, even more importantly subatomic level. The results of those kind of researches gave us the very foundations of modern technology in the form of transistors and computer memory, not to mention how that data help us in understanding the whole physical world around us. When researching and gathering the needed data, the prospect of general data measurements becomes significantly more difficult at the subatomic level. Two key principles emerge when looking into measuring energy data in quantum mechanics. One is the concept of superposition and the other is the process of collapsing state.</p><p><a href="https://en.wikipedia.org/wiki/Quantum_superposition">Superposition</a> comes from the notion that tiny particles such as electrons and protons can indeed exist in multiple states simultaneously! Crazy, right? This means that a particle can occupy spacial matter in two or more locations at the same time. Not only that, at that level, a particle can hold different properties at the same time.</p><p>The other key principle, measurement of a particle, can lead to something what’s called the <a href="https://en.wikipedia.org/wiki/Observer_effect_(physics)">observer effect</a> when the particle collapses into a single, more “stable” state while being observed (or measured). It’s easy to understand how this complicates any attempt of obtaining tangible and predictable data if a particle is changing its properties every time its being measured.</p><p>To further complicate things even more, quantum mechanics also introduces the concept of <a href="https://en.wikipedia.org/wiki/Quantum_entanglement">quantum entanglement</a>, an event in which two particles are connected in such a way that one directly depends on the properties of the other one. This directly alludes to the notion that in the process of measuring the state of one particle we can directly affect the state of an another one. What’s the fun part in this? The distance between two particles doesn’t seem to matter! With that we are knocking on the door of <a href="https://en.wikipedia.org/wiki/Quantum_teleportation">quantum teleportation</a>, but that is a subject to be explored more in the future (I’m pushing the limits of my mind enough as it is with this topic).</p><p>With all said, quantum mechanics provides the very foundations for quantum computing. Quantum computing is nothing more than a new way of ‘computing’ things and solve problems previously perceived as unsolvable, using the principles of quantum mechanics. When performing those computational tasks, quantum computing at its core uses quantum bits or qubits for short instead of the traditional binary (‘1’ and ‘0’) units of measurement in computer science. More on that in just a ‘bit’ (I’m not sorry for the bad pun).</p><p>It is important to note that although the unquestionable relationship between the two, quantum computing and quantum mechanics are completely two separate fields. The understanding of the principles covered in quantum mechanics are essential in understanding and developing quantum computing algorithms on top of the ‘general’ knowledge of computer science and mathematics</p><h3>What is a qubit?</h3><p>In 1935, <a href="https://en.wikipedia.org/wiki/Albert_Einstein">Albert Einstein</a>, <a href="https://en.wikipedia.org/wiki/Boris_Podolsky">Boris Podolsky</a> and <a href="https://en.wikipedia.org/wiki/Nathan_Rosen">Nathan Rosen</a> published a scientific <a href="https://plato.stanford.edu/entries/qt-epr/">paper </a>describing the phenomenon of quantum entanglement mentioned before. Since then the concept of quantum computing and even quantum cryptography has branched from the concepts of quantum mechanics.</p><p>A qubit represents the basic unit of informational data in a quantum computer and is directly comparable to the traditional ‘bit’ in the classic computer system. The main difference between the qubit and the classic bit is that the qubit can have a lot of states instead of just the two (1 and 0) of the classic bit. This is the concept of superposition that quantum mechanics teaches us. In theory, should help improve the computational speed of certain types of problems a lot. In a quantum computer, each qubit can be both a 0 and a 1 at the same time with a certain probability for each value. This, again in theory, can enable quantum machines to execute certain computational tasks in parallel which leads to reduced computation times.</p><p>The idea for using quantum mechanics for performing computational tasks is first pioneered by <a href="https://en.wikipedia.org/wiki/Richard_Feynman">Richard Feynman</a> in the early 1980’s. An interesting figure on himself, he introduced the concept of <a href="https://en.wikipedia.org/wiki/Nanotechnology">nanotechnology </a>and more importantly for this topic, the mathematical expressions describing the movements and behaviors of subatomic particles known as <a href="https://en.wikipedia.org/wiki/Feynman_diagram">Feynman diagrams</a>. The first demonstrated examples of quantum computing followed in the 1990’s using nuclear magnetic resonance (NMR) spectroscopy.</p><p>While the qubit is very difficult to work with by the nature quantum mechanics and simplistic maintenance of their quantum states is challenging to say the least, there is an ongoing effort to research all the potential benefits of it that can provide a promising area of research.</p><p>At the time of this writing, there&#39;s ongoing work by engineers and physicists to improve the overall robustness and ‘stability’ for practical applications.</p><h3>Use cases</h3><p>Quantum computing might seem like the clear winner when it comes to computational speeds against its classic computer rival with some calculations that would take the ‘opponent’ longer than the age of the known universe, a quantum computer would execute in a few seconds, but that’s not the case.</p><p>Not all calculations can be performed faster on a quantum computer, notably tasks that we take for granted in our daily routines such as media playback, browsing the Internet or typing a new blog article can, on average, take longer to be done on a quantum computer. The technology is still in its early stages of development with news about new funds and faster quantum computers (more qubits) coming out on regular intervals. Where the quantum computer struggles the most is in its practical implementation. Since it’s an emerging technology, many use cases for it have yet to be though of.</p><p>A few use cases where quantum computing might have a significant impact in today’s technology are the fields of cryptography, machine learning, AI and medicine. The field of cryptography keeps coming up in conversations and articles about quantum computing as something that seems to be the likeliest and easiest to find usage for. Breaking encoded data that was previously thought of as being unbreakable seems like something that would interest potential investors in this technology.</p><p>Machine learning and AI could see new optimization techniques that could cut-down on the times spent on training new models. This would lead to shorter turnarounds for new models based on new and updated data sources.</p><p>Simulations and optimized prediction algorithms can be brought into the spotlight with much more accuracy and discoverability of new, previously unthought of solutions. But, these are just some of the use cases that are talked about most in the scientific world, the most impactful use cases are probably waiting for the further development and adaptation of the general quantum computing field.</p><h3>Getting started with quantum computing</h3><p>Let’s get this right out of the way, one does not need a quantum computer in order to get into quantum computing. A <a href="https://en.wikipedia.org/wiki/Quantum_simulator">quantum computer simulator</a> would most likely satisfy most use cases developers might have.</p><p>To start of a journey into the field of quantum computing, developers must firstly cover the basic concepts of quantum mechanics such as quantum entanglement, superposition and <a href="https://en.wikipedia.org/wiki/Quantum_logic_gate">quantum gates</a>. While there are countless resources available on the internet, <a href="https://www.amazon.com/Strange-Story-Quantum-Banesh-Hoffmann/dp/0486205185">The Strange Story of the Quantum</a> (hence the subtitle) might be a good starting point. The book is quite popular in the quantum theory field and should be easily obtainable in paper, e-book or audio-book formats.</p><p>Learning a quantum computing-specific programming language or frameworks that can be used to experiment on quantum computer simulators should be the next step for anyone interest in this topic. There are a few programming languages specifically designs for quantum computing. The first one is Microsofts own <a href="https://learn.microsoft.com/en-us/azure/quantum/overview-what-is-qsharp-and-qdk">Q#</a>, the documentation for which is quite exhaustive. Familiarity with C# is also recommended if deciding to go the Q# route.</p><p>While the documentation for the <a href="https://www.mathstat.dal.ca/~selinger/quipper/">Quipper</a> programming language might seem a bit out of date, it can’t be argued that it is at least functional.</p><p>Like any process of learning a new programming language, experimentation is key. By experimenting and running tests on quantum simulators, one can form a better understanding how abstract concepts can be applied to in the new field. The process can be best described as comparing programming language paradigms such as object-oriented programming and functional programming for example. Both can achieve almost the same results at the end ,but to fully utilize them, one has to understand the way they are meant to be implemented and change their approach to problem solving accordingly.</p><p>Taking a dedicated course on the topic from one the popular teaching platforms such as <a href="https://www.udemy.com/">Udemy</a> or <a href="https://www.coursera.org/">Coursera </a>can provide a nice structured, step-by-step way to learn more and apply the gathered knowledge on various testing platforms or quizzes.</p><p>After grasping the basic concepts of quantum mechanics and quantum computing, learning about the various quantum computing algorithms should solidify any gathered knowledge about the topic.</p><p>Some of the more vocal algorithms in the world quantum computing are:</p><ul><li>Quantum gates: The basic building blocks for quantum circuits used for performing operations on qubits</li><li>Shor’s Algorithm: An efficient algorithms for factoring large integer values</li><li>Grover’s Algorithm: Used for fast searching through unsorted databases</li><li>QFT: A quantum version of the Fourier transform</li><li>QPE (quantum phase estimation): An algorithm used to estimate the values of scalar values in matrix equations used in many quantum algorithms</li><li>QML (quantum machine learning): A set of algorithms used for performing machine learning tasks such as supervised and unsupervised learning</li></ul><p><a href="https://qiskit.org/">Qiskit</a>,a popular framework for working with quantum computing, includes implementations of the before mentioned algorithms and many more.</p><p>Below is an example implementation of the quantum circuit that creates a superposition of two qubits done using the Qiskit framework:</p><pre>from qiskit import QuantumCircuit, execute, Aer<br># New circuit with two qubits<br>qc = QuantumCircuit(2)<br># Apply a Hadamard gate to the first qubit to put it into a superposition<br>qc.h(0)<br># Apply a controlled-not gate between the first and second qubits<br>qc.cx(0, 1)<br># Execute the circuit on a Qiskit statevector simulator (CPU &amp; GPU optimized)<br>backend = Aer.get_backend(&#39;statevector_simulator&#39;)<br>job = execute(qc, backend)<br>result = job.result()<br># Get the final statevector<br>statevector = result.get_statevector()<br>print(statevector)</pre><p>For those that have significant programming experience, they might have an easier time with picking up a new programming language designed for this topics, but nonetheless, the general path outlined in this section remains almost the same. The basics of quantum mechanics still need to be clear before moving forward with a career in quantum computing.</p><h3>Carrier opportunities</h3><p>At the time of writing this, there are significantly less jobs on the market for those seeing a career in quantum computing than there are for more conventional computer science technologies, but like with any emerging technology, this can quickly change and new job openings might be closer than we think.</p><p>There aren’t many places where quantum computing is being researched and utilized. The general perception is that that only large corporations have in their budgets room for research and development in this field. This is not necessarily true. There are smaller companies that seek to get their foot first into what might be the “next big thing”.</p><p>For those little job postings that are out there (again, at the time of this writing) there should be a few pieces of advice for those looking for applications. Without learning the principles of quantum mechanics and having a strong foundation in mathematics, especially linear algebra, the chances of landing a position are quite reduced.</p><p>Writing research papers on the subject and extensive networking are bonuses that can land a person a new career in a exciting new world of the qubit.</p><h3>Final notes</h3><p>This concludes a short first episode in what should be a series of article posts dedicated to the subject of quantum computing. Again, this article turnover has been significantly shortened thanks to the amazing tools such as ChatGPT.</p><p>It is important to note that the principles of quantum computing can potentially be difficult to understand and are often counterintuitive to what we are used to in the more traditional mechanics. The field is constantly growing and those working hard on it,are still trying to understand the full implications of quantum mechanics when applied to the world of computing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb8b4e53e74d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Developer Experience (DX)]]></title>
            <link>https://sdremthix.medium.com/the-developer-experience-dx-929ad944e776?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/929ad944e776</guid>
            <category><![CDATA[api-development]]></category>
            <category><![CDATA[information-technology]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[developer-growth]]></category>
            <category><![CDATA[developer-experience]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 17 Oct 2022 04:34:09 GMT</pubDate>
            <atom:updated>2022-10-17T04:34:09.185Z</atom:updated>
            <content:encoded><![CDATA[<p>Supporting developer inspiration rather than limiting it</p><figure><img alt="Bullpen on a laptop with shallow focus" src="https://cdn-images-1.medium.com/max/1024/1*qNFSk1m7RPLyWEvUYxrLYg.jpeg" /></figure><p>A lot goes into planning a products lifestyle, especially if that product is expected to have a significant commercial appeal.</p><p>I have to admit that I haven’t researched this topic before committing and writing a substantial portion of this article and to my surprise this topic has popped up all over the Internet. It’s seems like an interesting topic to explore and you can checkout a few of them right <a href="https://www.getclockwise.com/blog/what-is-developer-experience">here</a> and <a href="https://developerexperience.io/practices/good-developer-experience">here</a>. On the other hand, while those articles and search results yield a great deal of input in what goes into developer experience, I’ll try to provide my thoughts on the subject from a bit different angle. Since there seems to be a slight shift happening in the software development world with the focus hopping from a developers workflow to release schedules and needless product complexity, it only seems fitting to explore what aspects go into the topic of this article.</p><p>Planned almost a year ago (with the seed being planted God knows when), the idea for this article has only grown in my mind in the past few years in the software development industry and is eager to escape.</p><p>At what point did the industry go wrong with this if we can even call it a wrongful direction? Can we go back to improving the development process without compromising the publishers market’s target values and goals? Can the focus shift back to providing developers the necessary tools to make their coding process a bit more enjoyable?</p><p>What follows are some of the areas that I think, define a product usage experience in no particular order.</p><h3>Creating meaningful API’s</h3><p>The application programming interface (API) is our way to provide a meaningful communication channel between two entities. It defines a common language (contract) that both parties understand. What can clog-up that communication is an abundance of access points that the service entity tries provide with it’s API contract.</p><p>The exposing of only the bare minimum functionalities to the target consumer service through it’s API can result in better justification of the existence of the API in the first place. Any API that leaks it’s inner workings through the service or error messages, other than a potential security risk, also lowers the confidence level of the consumer developer in the product as a whole.</p><p>Less options lead to more focus and thought of what a complete developer experience should be and finding the right balance of complexity and usefulness with ease of use. If we are afraid that the lack of options in our API’s would sway away any interested consumers in a ever-growing, feature-rich market, we are losing the sight of makes a product good in the first place and that’s the products reliability and usefulness of the solution it tries to provide.</p><p>New features tend lead to product fragmentation and the initially set goal for a product slowly disappears and merges into a messy blob of functionality that has lost its rigidness a long time ago. If made to choose between two competing solutions, guess which one would the developers more likely gravitate towards? The one with the leaking, overly-complex and option-saturated API or the one with the bare minimum, tightly- developed and tested one that also gets the job done with much less overhead and lower learning curve?</p><p>While the task of implementing new API’s is usually assigned to the first available developer, it should be at least questioned if the new API’s are meaningful and intuitive enough to be simply assigned to anyone. What is intuitive for one person can be detrimental to another. API documentation is usually left to be drafted by more experienced developers and then checked by other ones for any errors or inconsistencies. Overly complex API’s should be avoided when possible and alternative solutions proposed. Never lose track of the developer for whom you are designing the API in the first place.</p><h3>Peer testing</h3><p>So you’ve made an interesting and exiting new framework, API, service and are eager to hear what the development community thinks of it. You’ve published your project on various social media posts, developer channels, blog posts, etc. and now you wait for the feedback. The replies start coming in and they are overwhelmingly positive! Praising the new product you’ve published and the problem solution it provides. Congratulations! You made a great first impression with your project, but this should only be the start. Namely, depending on the size and complexity of your project, the developers from the developing community might not had enough time to fully integrate your solution into their projects or test to the extent needed for a fully production-ready solution.</p><p>One important thing to note is that, while not to be underestimated, the developer community providing the feedback might be a very small percentage of the end-developers that could end up using your product. The number of initial peer testers depends on various things: the number of existing developers following your work, your previous success in making developer-consumable products, publishing channels, subscribers, groups, and so on. One thing remains the same, not all of the developers that will consume your product will leave a meaningful review or feedback to help you create a more impacting end-developer experience. The vast majority of the developers that use a product simply don’t extend their impressions about it beyond the office-setting, developer small talk. This is a sad state of things in the software development industry.</p><p>The most likely scenarios to happen are the ones where developers don’t provide any kind of feedback, they just use a product or they don’t and in the case of new API’s for a large existing framework, they most likely don’t have any options left. They are simply boxed-in by the over-encompassing shadow of the much larger framework, so they have to use the new API’s or risk much time, effort and cost of switching into a completely new framework. When possible, it’s always a good thing to implement some kind of usage metrics to monitor the said activities of the end consumers. Google/Firebase <a href="https://firebase.google.com/docs/analytics">analytics </a>are a good starting point, but implementing a custom solution is also a valid option if the development budget allows for it.</p><p>Since we are already here, we can perhaps question who the developers who provide the feedback are. We already mentioned that only a percentage of the target developer community provides any feedback in the first place. Well, who are they? What’s their previous experience? Do they simply follow the latest developer trends or do they give a bit of thought into what products they end up dedicating their time towards to? How much time did they spend using your product? There are developers out there that look forward to frequent changes and see migration guides as something challenging and something to sharpen their skills with. The truth is that not all developers have the time or patience to constantly ‘tinker’ with hard to justify changes. If we constantly look for feedback in the same peer groups, we could possibly risk losing a more objective oversight into our product. An analogy to this could be if we would constantly ask for feedback from our friends and family members about any work we do. You’ll admit that chances are that we’ll receive a biased opinion and the checks end up being picked up by our product.</p><h3>Testing, testing</h3><p>The only way we can guarantee that our product works is to have a bunch of test suits to assert our products functionalities against to.</p><p>It should be obvious at this point that only stable and well tested solutions are the ones that even stand a chance to make an impact on the market. No matter how good your solution or big the problem is that it tries to solve, if it can’t be reliable or deterministic at the end, it will always be questionable for the users to use it outside an experimental and proof-of-concept environment.</p><p>Any front-facing API should be carefully tested with each change to ensure that the existing functionalities don’t suffer from hasty-executed decisions made in conference rooms. Unless that was the intention, in which case alternative approaches should be communicated to the end users as breaking changes. Braking changes should be kept at the bare minimum.</p><p>Is your product often used alongside other third party solutions in some sort of a symbiotic way? May it be the OS platform or development IDE? What is commonly omitted when executing testing procedures is the validation that a product works well with some other third party solutions. If your product requires a specific set of third party solutions and system requirements to be able to function in a proper way, make sure to include profile testing for those requirements as well in your development pipeline.</p><h3>Forced updates</h3><p>Are frequent and regular release schedules necessary? Sure, if we have meaningful contributions to add. If we constantly think of new ways to improve our product, may it be with features, extensions, fixes or optimizations, we end up keeping ourselves busy. But, why should we be working on an already pretty much complete solution when we could shift our focus onto something else? Not every product needs to grow into a gigantic tentacle monster of endless and hard to maintain features. Some just need to do one thing and do that thing very well. At some point a product’s update release should phase out into simplistic, but regular maintenance cycles diving into more substantial updates when absolutely necessary for the product to remain relevant on a market.</p><p>While frequent updates are almost always welcomed and seen as a sign of developer dedication to a product, they can also be seen as a sign of untested, unpolished or even bad software practices, especially in the case of frequent patches. The developers that consume a service will loose confidence in the publisher and look carefully before trying out their other products in the future. This can be very hard on emerging companies and individuals that aim to make a name for themselves in which case every step should be carefully planned and executed.</p><h3>Doing one thing GREAT</h3><p>It’s easy to get carried away with creating a feature-rich product to remain relevant in a competitive market, but time and time again it has been shown that at some point the complexity becomes simply too large to be maintained and each new update becomes a pain.</p><p>If the entry point into a product is difficult, or worse yet, if the learning curve for it is very steep right at the beginning, we risk losing a lot of potential customers. This if obviously tied to the product domain. Not every domain has an easy first step to jump into. Being the definitive tool in the business can also open a lot of doors, not the same doors as for a feature-rich product, but a few doors nonetheless. So, it’s always good to keep one eye out on that learning curve.</p><p>A significant amount of products have outgrown their initial cocoons over their lifecycles into something unrecognizable with barely any traces of the original spark from which everything came to be in the first place. This is by no means a bad thing and it can be almost considered as expected giving a long enough lifespan of the product and possible shifts in industry trends that happened along the way. How many sales pitches for new API’s, frameworks, applications start of with including the keyword <em>lightweight</em>? Lighter than what? The previous industry-leading software tool? Most likely… There has to be a reason why the promise of something being a lightweight version of something else seems so alluring to the consumer market. Most of the consumers are focused on their end-product and finding a fast way to ship that product and the prospect of wrangling of some larger solutions doesn’t seem like a valid enough option. There’s a reason why so many bare-bone text editors keep creeping up on many <em>favorites </em>lists of software tools. The hassle is real and we aim to avoid them as much as possible.</p><p>If we take a look at startup companies, we would see that they usually focus on one thing and that their work mostly consists of trying to make that one thing excel until another company buys them out.</p><p>The topics mentioned in this article should at no point interfere with business models of others. Carefully crafted project outlines, roadmaps and business models will most of the times be enough for a product to survive and be kept alive long enough to grow its audience.</p><h3>Good documentation</h3><p>Most of us have heard the saying that <em>6 hours of debugging can save us 5 minutes of reading the documentation</em>. Or something similar at least. As developers we often don’t cultivate the habit of reading through a product’s documentation when engaging in the development process other the occasional venture in the setup-and-install entry points of the said documentation. Is there a particular reason why we do that? Perhaps we were trained by less-effective documentation pages over the years, resulting in our loss of confidence in the very same documentation or are we just used to as developers in the early stages in our developer careers to not put as much effort into writing meaningful documentation for our own projects?</p><p>Documentation like any written medium, can either provide satisfaction to the reader or not. A good product documentation drives the potential users through it’s contents and exposing all the relevant data and steps to be taken for various scenarios a user could potentially find themselves in when using a product. Clear and to the point without or minimal ambiguity what an action can do is the way to a developers heart. If users start raising issues for the documentation not being expressive enough or entirely missing key and important aspects, it’s a sure-tell sign that the documentation needs to be updated to address those issues. Developers are often used to browse though various discussion boards, issue trackers long before checking out the product documentation as the first go-to destination for any issues. It’s a habit formed only by bad or “dry” documentation experiences beforehand. It’s interesting to note that most of the accepted answers in the said discussion boards are direct quotes and links from the products documentation pages. This only shows that the documentation is either difficult to navigate through or plain laziness of us as developers to query the documentation first before asking other developers for help.</p><p>Writing good documentation should be considered an essential part of software development. Usage examples and short, to-the-point descriptions can go a long way in supporting our main product. A lot of heavy lifting in generating meaningful product documentation can be achieved directly from a project’s source code via code commenting. A good example of how intricate and helpful code commenting can be, can be found on the Go programming language commenting resource <a href="https://tip.golang.org/doc/comment">page</a>.</p><p>There are scenarios where we as developers are usually provided with an existing solution by some other entity in our organization and are left to the mercy of the the same entity if they have skimmed thought the solution’s documentation and found it to be of satisfying, developer-friendly experience levels.</p><p>If we are given the task of writing the documentation pages, we should also be sure to provide usage examples in our documentation and, most importantly, test those examples to avoid any confusion and problems potential developers might experience when using our product.</p><p>We should aim to write our documentation as a sales pitch for our product or service.</p><h3>Avoiding contradicting yourself</h3><p>What happens when developers begin with course-correcting their product or solution year after year?</p><p>We heard many tremendous announcements of new releases by various product companies, that promised a lot of things, guides, recommendations only for the next years announcements to contradict all of it. So what happened in that one year? Did the industry change? Perhaps or perhaps the lack of research and thought put into a solution proved to be less adequate and the real-world usage data didn’t align with the expected one, requiring new adjustments to be made. We all make mistakes and that is as natural as it gets. Course correction can almost be expected nowadays. Making a habit of those adjustments is what should be avoided. Being careful when making grandiose announcements is strongly advised. There’s only so much that developers and whole companies will commit themselves to when it comes to constant migrations and course corrections, until they start exploring some other, most likely <em>lightweight </em>alternative solution.</p><p><em>1.0</em>, <em>stable</em>, <em>production ready</em>… A year after those claims, significant issues are still present or they’re being updated on a regular and frequent schedule. Those seemed more like alpha versions now, a year later, don’t they? It is important not to mislead developers into thinking they could include a product into production development when the opposite is most likely. Be sure to label your product correctly or as <em>experimental </em>if not covering the very basics of production deployment, stability.</p><p>A significant amount of developments are hungry for the next thing and they have the time and money to experiment with it. What si more likely to happen is that long term supported frameworks, libraries and so on are preferred and usually implemented in large scale projects. For greenfield projects, on the other hand, the shiny new thing might be considered.</p><h3>Making an ecosystem of services</h3><p>When an individual or corporation provide multiple services and products, an idea is formed for creating a whole ecosystem for those services. What would be considered as a service ecosystem?</p><p>A service ecosystem is a system consisting of at least two sub-services that are “aware” of each other. That awareness comes from the notion that they share common gateway elements that allow them to communicate with each other in some capacity whether that being a common API, design system or some other communication protocol. Do you have a whole existing set of services and products already made? Why not make them communicate in the same language. Teach developers what to expect from your product. A go-to example would be the Apple ecosystem.</p><p>What an ecosystem of services provides is a way for the consumers to utilize a sandbox of multiple services that allow them to execute any task without the need to include a new third-party solution at any stage of a tasks execution. What can be potentially considered a downside is that the sandbox paradigm leads to problematic inclusion of other external services when the ecosystem is incomplete or plainly missing a crucial part for a complete workflow and the potential difficulty of exiting that same ecosystem if it proves to be too enclosed.</p><p>The exiting from an ecosystem is a two-edged sword. One would ask, as service provides, why would we ever want any customer to ever leave our ecosystem? Too much of a good thing can be hurtful. If the ecosystem is not sustainable in the long-run, any “waves” that manifest as updates and changes to the previous, well-established properties of the said ecosystem can lead to some unforeseen issues with the consumers. If those waves are big enough, the service providers risk alienating any loyal users and drive them towards a competing service or worst yet, a competing ecosystem. Maintaining the fragile balance between maintaining existing values and introducing new ones is an art form in itself. Considering evolution rather than revolution when introducing changes to the ecosystem has proven successful for many large service providers over the years.</p><h3>Clear vision</h3><p>At the end, nothing matters more than having a clear vision what a product is, where its extended goals(development roadmap) are and where it fits in a very competitive market. Microcorrectins can go only so far, the long vision is what sets the foundations for something more lasting.</p><p>If you don’t have clear vision for your product, don’t expect the developers to have it instead. Don’t let developers drive your product with unnecessary demands that directly interfere with your vision. Acknowledge them and put those ideas on the backstack or into a complete new product if the ideas are too far of your intentional plans/visions. It is what it is. They can end up using it or not. You’ve already provided them the developer experience they were looking for.</p><p>We reached a point where we can finally say why DX is as important as, say UX is. DX drives the developers and in general makes their jobs a bit more easier and by extension they will be much happier and willing to use and even promote our product to other developers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=929ad944e776" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The importance of having a deadline]]></title>
            <link>https://sdremthix.medium.com/the-importance-of-having-a-deadline-c65391e76106?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/c65391e76106</guid>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[developer-productivity]]></category>
            <category><![CDATA[deadlines]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 05 Sep 2022 18:44:50 GMT</pubDate>
            <atom:updated>2022-09-05T18:44:50.252Z</atom:updated>
            <content:encoded><![CDATA[<p>Similar to the main subject of one of my previous<a href="https://sdremthix.medium.com/programming-for-obsolescence-dc8b6502546"> articles</a>, just the mere mention of the word “deadline” sends cold chills down most of our backs. Why might that be? Why do we give so much power to that word? Most of the topics explored in this article are from my point-of-view as a software developer, but much of it can be applied to other industries and day to day activities.</p><p>In this article, I’ll try to avoid the topic of mismanaged time and the proper ways to determine a task’s completion time as much as possible. These subjects fall outside the scope of this article and can be written about in length all by themselves.</p><h3>The power of a deadline</h3><p>So, what gives a deadline the power it has over our lives? Why does that term come with so much weight and is almost never pleasantly received by us?</p><p>It’s safe to say that we have been presented with some tasks with targeted deadlines at some point in our lives. Can you remember the first deadline you’ve ever received? Was it at your first internship job when you were assigned a task by your superior? Or was it in high school when your whole class had a written assignment that needed to be turned in by the end of the semester? Perhaps it was even earlier than that, when we were just children playing in the playground and our parents rigorously told us to “be home by 8 PM” or when to finish brushing our teeth and get ready for bed? The truth is that we were attached to some form of a deadline from our earliest ages. Could you recall what it must have felt then, being told to accomplish something by a specific point in the future? That feeling was most likely some form of fear or anxiety. Fear of making our parents upset. Fear of getting a bad grade. Fear of losing our jobs. The last one perhaps resonating quite loud in our adulthood.</p><p>So, from the earliest point of time in our lives we associate deadlines with something bad happening, perhaps to us or someone close to us. Something that carries the weight of consequence. No wonder why we shroud when we hear that word. We immediately associate it with something negative happening. From the point of receiving a deadline we tend to focus on the negative consequence of not meeting it. But what about the positive things about it? What happens when we meet the deadline?</p><p>Well, in that case there’s usually some kind of reward waiting for us. Maybe it’s the reward of having healthy teeth because we rigorously brushed them or a good grade or a promotion. If that’s the case, why don’t we associate a deadline with a positive outcome? Maybe because the number of negative outcomes outweigh the positive ones. That could be very well be true. On one hand we have the negative outcome of not meeting a deadline with the potential of being demoted or even losing a job with the positive outcome being a new task and new deadline or a pat on the shoulder for doing a good job. Yes, in this proposed scenario the consequence for missing a deadline is quite disproportionate to the sight of meeting it. Like a well-oiled machine, with time its expected from us to deliver as much as possible tasks with little to none missed deadlines in order for the machine to keep going. Failed tasks make the machine screech to a halt and depending on the size of the machine (e.g.,the budget for a project ) it can have severe consequences not just for us but our colleagues and even the whole company that we work for. So, the negative outlook to the term is now a bit more justified?</p><p>No, not exactly. In the extreme example mentioned above the idea that one person alone has the impact on the whole company is a bit exaggerated. I’m not saying that it’s not possible. Over the time we have witnessed that the decisions made by individuals have severe impact on countless other people, but the idea that an ordinary employee has that much power is less likely. Companies nowadays have redundancies in place that prevent that kind of thing of happening. Yes, missing a deadline can lead to a hiccup, but the machine continues to work as expected. What about outside the scope of company employment? What about our personal lives? On the other hand, a deadline is not exclusively attached to our jobs.</p><p>The same can be said in the case of our personal lives. We are expected to complete our day-to-day tasks and if those are not met in a predefined time, then we are presented with some kind of consequence the severity of which can range from something minuscule like missing an episode of our favorite TV show to something more important like missing a date with our loved ones. And this is true for all types of deadlines (yes, there are multiple types). Don’t think that self-imposed deadlines have less impact on our lives. Missing a few of them can lead to losing the confidence in ourselves when we assign new tasks. Deep down we have the small ball of doubt that keeps growing with each deadline we miss.</p><p>Could it be that we involuntarily give a lot of power to deadlines? We are feeding them with our angst, our lack of commitment or even the most heinous lie we keep telling ourselves:” I’ll do it tomorrow.” How can we improve in this particular area?</p><p>The key for building ourselves up might be in the way of how we approach new tasks and deadlines. Instead of looking at them as something standing over our shoulder, we can look at it as something that is very much finite. Something that has an expiry date attached to it. It might not sound very rewarding, but it can very well be. Can we, at least, agree that the proposal of something being finite seems much better than something lurking about without the prospect of ever leaving? Something that you deep down know has to be done, but with no deadline attached it will always hang around us for days, months or even years. The thoughts of: “I should do that…at some point of time.” or “I should get around to doing that pesky task…”. seem a bit too open-ended to be something definitive. Imagine saying those lines to someone else. They almost feel like lies. If we reword those lines into something more deterministic like: “I’ll have up until next Friday to finish this.” and “This task has to be done today.”, we knowingly assign timed values to those tasks. We have already dedicated part of our time to execute them. In some sense, we made an investment into something and it’s almost always only up to us to cash in on that investment. If we don’t do the tasks, our investment is gone and it’s not coming back. The only way to “fix” it is to make a new investment in it, but now our new investment comes with the label and burden of the previous failure (or multiple failures if we continuously prolong any completion). Doesn’t seem too fair on the new tasks now, does it?</p><p>There are certainly a lot of ways around this, the most pervasive one is to stop lying to ourselves. If we constantly defer finishing a task, deep down we know that a new deadline on a stale task wont make it fresh. When recognizing if that might be the case, taking a few steps back and analyzing the task itself can lead to its final completeness. Evaluate and re-evaluate. Maybe after so many prolongations, there’s no point in doing a task anymore or maybe some external factor is blocking the execution. At the end, it’s only up to ourselves if we are going to address it or not. It’s our time that we invest.</p><h3>The illusion of not having a deadline</h3><p>Imagine this: you wake up in the morning, it’s Friday and the sun is shining bright through your window. You have breakfast and get ready for work. Weightlessly, you go to work. The long, dreary trip to work, today, seems unusually short when you arrive at your work place. You sit down at your work desk and just start working on your planned tasks. But there’s something different than all the previous work days. All the tasks for today have no deadlines attached on them. Today, no one’s looking over our shoulders and checking their watch in a nervous manner.</p><p>I believe that we would lie to ourselves if we wouldn’t feel a bit more relaxed when there are no angry clock eyes on us all the time. So many things that we do in our lives have some sort of a slowly closing gate that separates us from our goals. A good gauge of the closing gates speed and if we can make it in time to cross it, can lead to us eventually crossing that gateway or to sit back and watch it slowly being shot close before our eyes.</p><p>What we usually don’t understand is that not having an initial deadline for a task, that doesn’t mean that one will not materialize itself at some point in the future. If (usually its more “when”) that happens, it puts us in a very uncomfortable position and something that initially was something minor or seemed as unimportant in the sense of priority of execution, turns into a monster waiting for the opportunity to jump and attacks us. Every task grows over time, weather we like to admit it or not.</p><p>Tasks are created and assigned for a very good reason. Not having a tight deadline or having one that, to us at least, seems very far of, doesn’t mean that the given task does not need to be completed. Rare are the cases that long prolonged tasks and those of lower priority are, in the end, shelved or even cancelled. As developers we should approach every task with the same goal in mind. When we sit down to work on a task, we should also aim to complete that task in a “reasonable” amount of time. A deadline should only exist as a marker that, if reached without completing the task, reminds us that something went very wrong and that we lost the window of chance to actually have some kind of input in the tasks execution. In many cases the task will be resigned to some other person, but there will always be the mark that something wasn’t completed in time.</p><p>Like I’ve previously mentioned, by assigning a deadline to a task we make a sort of commitment and investment in actually seeing it through. We can all agree that checking out a task from our task sheet feels good? Certainly feels better than constantly opening up a task sheet and day after day seeing the same task(s) aimlessly sitting there with no special time assignments to it. It has a two-fold meaning. One: the task is not important enough to have any urgency about it; and two: the task doesn’t feel personal. The first one is pretty much self-explanatory, but the second one requires a bit of explanation. Every task that is assigned to us has to feel somewhat personal in order for us, as the task’s executors, to register it. It can have our name attached to it, but that doesn’t make it personal. What are the benefits to completing it? What are the consequences for not doing it ever? Indeterminate tasks can’t have any of those things, because of their open-ended nature. As such we don’t feel connected to them and see them more as some notes left at our desk for the person in the previous shift (if that’s the kind of job) or topics for discussion because, at the end, it somehow has our name attached on it. An example of a non-personal task can be: ”Learn to play piano”. To whom is this task meant for? By when should that person learn to play the piano? Today (not likely), next month, next year, by the end of the decade? By comparison: “Learn to play the piano in a year”. OK, now we can work with that. It’s not ideal, but at least we have a point on the horizon we can aim for.</p><p>Even if you work as a freelancer or have your own company or projects, its a good thing to have some concrete deadlines for all of the tasks you need to complete. By doing so you’re developing a habit and the highly important discipline that can elevate your professional career.</p><h3>Unrealistic deadlines</h3><p>An unrealistic deadline is any deadline that doesn’t take in account the time available of the person doing the actual work. You can have 10 tasks with seemingly realistic deadlines, but if all 10 of them are assigned to a person or team that has already too many tasks on their hands, the new deadlines jump from being realistic to being something in the realm of fantasy. The same goes for the tasks in our personal lives. If we already have a whole queue of tasks waiting for us for today, adding new tasks will most certainly make the previous task feel more cramped. What can we do to avoid those scenarios?</p><p>Your work discipline and objective self reflection, play one of the most important roles here. It’s not out of the realm of possibility that our manager, client or whoever presents us with a seemingly impossible deadline for a task. How many times have we heard the dreaded sequence of words: “It must be done by <em>insert arbitrary short time-span</em>”? Sounds familiar, doesn’t it? And how many times did you counter that sequence of words with mentioning how the proposed time-span falls outside any realistic scope of the projected time for completing the said task? As professionals, it’s not unreasonable to presume that we should have some input about any proposed deadlines. We should be presented with all the information needed in order for the seemingly tight deadline to be justified in our and the minds of our colleagues (if we work in a team environment). If no additional information is provided to us, it’s up to us to ask for more information. Without additional information we might not know that the proposed deadline is defined by some other, external event. Perhaps there’s a fashion show scheduled and the promotion for it must start at the same time the deadline expires. Maybe some big sporting event, like the championship finals, are just around the corner and it’s crucial for a console game to be shipped to promote itself and that event. Simply put, there are things that are outside our or the management’s control and as that they come with some predefined timetables that may or may not be compatible with our internally projected task deadlines. But we can’t know about those things if we first don’t ask. Be sure to gather as much information as possible in order to calculate and present a more realistic estimate for the task completion. Perhaps some information was miscommunicated and, in actuality, there is room for some time extension or perhaps the tasks priorities have been changed and now you do have some additional time. If not, we can always calculate our own time estimates.</p><p>The more your own time estimate differs from the one given by the manager, the more it points to potential problems and it should be dealt with as soon as possible in order to obtain more realistic expectations about the task completion time. Present your concerns and be sure that you have gone through all the projections in order to back up your own time estimates with the necessary data to convince those with the power to change deadlines to actually change them. Going in front of your higher ups without any convincing data that backups about your unrealistic deadline claims, can be looked at as amateurish and it can propose a concern for your task completion estimation skills.</p><p>In the same vein, we should be careful when we assign tasks to ourselves. We can be great work managers, but lousy managers of our personal time. This is when our ego comes out. We like to think that we can accomplish so much during the day, which can very well be possible, but, in general, we don’t put so much effort in time-estimation for the tasks we give ourselves. We would start the day off with one list of tasks and the day with an unrecognizable list at the end of it. What we can do is to consider taking a few moments to think about a task and its relationship to other tasks on our to-do lists before actually adding to it. How compatible that task is to other tasks. What I mean about this is that some task simply don’t “work” together. Going to the end of the city to buy some item that realistically will take around 2h of our time is not compatible with sitting home and cooking dinner. Buying an item takes around 10 min. in the grocery store, but the mere trip to the store and back takes a lot more, so somewhere that time has to cut-in.</p><p>There shouldn’t be such a thing as an important task (not to be confused with task priority). Every single task carries the weight of time it takes us to complete it. It wont magically disappear, no matter what we tell ourselves. The lower priority tasks will be shuffled at the end of the queue with respect to their deadlines.</p><h3>Stretching and elastic deadlines</h3><p>Parkinson’s<a href="https://en.wikipedia.org/wiki/Parkinson%27s_law"> law</a> states as follows: “<em>work expands so as to fill the time available for its completion</em>.”</p><p>The idea of the ever-postponing task as a realistic option in any kind line of work can be like a mosquito that buzzes around our heads in a warm summer afternoon. We shake our heads and swing with our arms to make the mosquito go away, but after a short absence, the mosquito comes back and continues with the aggravating buzzing. Its only when we decide that we had enough and focus on the nuisance that is the mosquito, that we are actually making some progress in dealing with the problem.</p><p>Postponing tasks only leads to them being grouped together with new tasks and cut into their respective deadlines. What we now have are some sort of task vampires that, not only crave for their own execution times, they actively bite into the time needed for another task’s execution. Instead of a task living in its own little time bubble, we have the situation of multiple tasks sharing the same bubble. At some point that bubble has to burst and we don’t want that to happen.</p><p>“What about multitasking?” — you might ask. I’ve read somewhere about a comedic definition for multitasking: “Multitasking — the ability to do multiple things at once, badly”. This is not at all the case for most of us. We do multiple things at once almost all the time. The thing that is important is to distinguish heavy, important tasks from those that we do almost automatically. Not every task demands the same amount of our concentration. Doing multiple smaller, less thought provoking tasks at once, is very much feasible to most of us, but doing more engaging tasks goes in counter-productivity. There’s only so much of us available to effectively execute anything. Perhaps we sometimes overestimate our abilities and what we end up doing at the end is just making a mess of everything with no task being executed in its fullest.</p><p>In the case of one of my own personal projects, I had a list of tasks with assigned deadlines for each one. One after another, I’ve completed most of the tasks well inside the proposed deadlines, but there was one task that I’ve prolonged for so long that almost unknowingly it missed its initial deadline. Since I was the only developer on that project, I simply assigned a new deadline for that task and continued the work on other tasks. Once again, the problematic task has reached the deadline marker with no sight of its completion. A deadline renewal was in order, but I didn’ t do that. No more time reassignments for that task. What I did is to re-evaluate the task’s importance in the project and the priority for it. Obviously something was problematic with that task from the beginning that made it so difficult to be executed. After analyzing the task again, I came to the conclusion that that task was too complex by nature. Not difficult in the sense of the problem it tries to resolve in the project, but by the mere heavy lifting it tries to do by itself. Instead of the task being easy to understand and very contained, it depended on so many other things to be done beforehand in the preparation steps that in order for the task to be marked as “<em>complete</em>”, basically the whole project needed to be complete. This wasn’t an issue with the initial time estimates and priority, this was an issue with the tasks definition. From the begging that task shouldn’t have existed. It should’ve been split into several smaller tasks that could be more easily integrated in the projects development lifecycle. After completing the re-evaluation process, new tasks were created the development workflow continued.</p><p>This can frequently happen with difficult tasks. And don’t make a mistake, the task is difficult since it takes so long to complete. No matter how simplistic a task is, every time we don’t complete it in time, by default, it makes it a difficult task. It might not started its life as a difficult task, but we let it become one by not addressing it in time. If a task or multiple tasks start being renewed after reaching their deadlines, we should look at their definitions in more detail. Splitting a larger task into multiple smaller ones, could be a much better way with putting us a bit closer to finishing it. This can most definitely be applied to so many things in our lives. In general, we should aim to complete as many tasks today in order for tomorrow to have room for other things.</p><h3>Task priority</h3><p>So many times, in agile development we come across tasks that are constantly being re-evaluated and their priority re-assigned. A number of tasks that seemed important at the start of project turn out to be not important at the end. Some even being discarded without a single minute of actual development time dedicated to them. This is more common when working on large projects. A lot of times our ideas and thoughts about a project tend to be different at the start then those at the end of a projects development. This is simply because we didn’t have the same information at the start that we have at the end of a project. Some unforeseen circumstances such as external influences (client change requests, industry changes, personal life, etc.), make it almost impossible to have a 100% oversight of a project from beginning to the end. The waters are somewhat murky at the beginning, but we don’t see them as such because we are focused on the horizon instead. Don’t get me wrong, there are absolutely projects that have stuck with their initial plans and projected timetables from start to finish, but a lot of them don’t have that privilege.</p><p>A task’s priority is determined from its relationship between other tasks, their overall contribution to reaching a certain goal and, of course, a deadline. A task that is crucial to the further progress of a project with a tight deadline is most certainly going to be looked at as a <em>high-priority</em> task.</p><p>It should be noted that a task’s priority is not fixed in place, but it can change over time. A task that starts out as <em>low-priority</em> can become <em>high-priority</em> just because it’s deadline is close to being reached.</p><h3>Mark it on the calendar</h3><p>This might sound not so important, the simple event of marking a task in the calendar, but can have a lot of benefits. If the calendar is placed somewhere noticeable in our environment, maybe as an app (on our phones, laptops and computers) or physical calendar on our desk, just by glancing at it can remind us of some of the responsibilities that we have to do.</p><p>Day to day we are juggling a lot of things in our heads. It’s a busy traffic there all time and it’s quite unreasonable for anyone to expect that we don’t forget about things from time to time. It’s pure human nature. We, by default, prioritize and shuffle things to remember all the time and as result of all that shuffling about, some things get to be misplaced. New information comes pouring into our lives and its hard to filter everything and avoid the information clutter that inevitably will arise. What can we do to avoid this? The simplest thing is to just write everything down.</p><p>The unceremonious event of just writing something down, elevates that subject from the ground level and the rest of the information flood. That elevation might not be much, but at least we have dedicated a little bit more of our time, effort and promised to ourselves that something should not be ignored or forgotten about in the near future. This might sound a bit “old fashioned” to the younger readers out there, but studies have shown that writing something down, the physical act of using pen and paper and not just typing it in a note taking app, could result in a bit more insurance that something will more likely be committed to our long term memory. In addition to the last statement, something should also be said about writing everything in our own words and understanding. Like in how we would explain a topic to some other person. It might be easier for us to understand something in our own words and thoughts than someone else’s. I realize that this might fall outside the subject of this article, but I just felt that some readers would be upset if I wouldn’t mention this.</p><p>When giving tasks their deadlines it is important to not forget to give them a little bit of “room to breathe”. Grouping multiple heavy tasks with their deadlines lining up right after one another can result in concentration overlaps. An example of this would be something like finishing up an important 3h meeting and following that with a 3h task and then a 2h mentoring session. This might fit in the 8-hour workday. It looks very “doable”, sure, but what we fail to realize is that the meeting might have been very exhausting and it takes us around half an hour just to get on the right concentration level for the next 3h task, but at that point we might be very well behind schedule and start to pick up pace to finish the task. Let’s say that we managed to complete the task on time and we jump in the mentoring session. At this point we are even more exhausted than we were after the meeting. Needles to say that it now takes us a bit of time to concentrate on mentoring. And it goes like that on and on. Giving a bit of time to prepare for a task can be helpful in rising our focus and concentration levels a bit. This also goes for any “success rush” we might have after we’ve achieved something important.</p><p>Today we have powerful tools that help us manage our tasks and even visualize a task and its priority for even more clarification. Seeing a task marked red for a day can immediately tell us that that task needs to be dealt with asap.</p><p>Putting a calendar somewhere where we can see it easier, helps us to better remember that there are still some unfinished tasks left to do. This can also put a lot of pressure on us if we have a lot on our shoulders.</p><h3>Outro</h3><p>Deadlines help us in reaching our goals by forcing us to think about the tasks. If a task is reached in the set amount of time, great, but if not, we now have to reevaluate everything around that task. It’s very easy to assign a deadline to a task when creating one and we can easily fall in the trap of assigning arbitrary values for it that can come out of nowhere and cause potential panic. Taking a bit of time and looking at the overall picture where the task fits in, can save a lot of potential frustration later on.</p><p>The simple truth is that we don’t hate deadlines, just unrealistic ones.</p><p>What are your thoughts on deadlines? Do you have a habit of assigning deadlines on your personal projects too? Looking forward to hearing your thoughts. Thank you for reading.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c65391e76106" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Project provisioning/handover]]></title>
            <link>https://sdremthix.medium.com/project-provisioning-handover-caec9673b02b?source=rss-741be2c6b322------2</link>
            <guid isPermaLink="false">https://medium.com/p/caec9673b02b</guid>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[workplace]]></category>
            <category><![CDATA[project-provisioning]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Srdjan Delić]]></dc:creator>
            <pubDate>Mon, 05 Sep 2022 18:37:03 GMT</pubDate>
            <atom:updated>2022-09-05T18:37:03.205Z</atom:updated>
            <content:encoded><![CDATA[<p>the art of being a professional at the end</p><p>The average software developer has quite a few projects that they have worked on or are working at the moment, but a time comes to pass on that project to another pair of hands. As true professionals of our craft, it is admirable to make that extra step and solidify a positive work relationship between the two parties, the project sender and the project receiver. As someone who’s been on both ends, I can say that it can be very <em>unprofessional </em>when crucial parts of the project are not disclosed or simply do not exist.</p><p>Over the years, I’ve gathered some helpful insights, tips and guides on how to make process of handing over a project as painless as possible. So let’s not waste anymore time and just dig into them.</p><h3>Setup instructions</h3><p>First things first, providing a clear, step-by-step textual or even a video guide on how to build and run a project (if in a runnable state at the first place). Going the video route is, I admit it, a true stretch, but in this day and age it could be a lot more useful than a instructions manual.</p><p>Like deciphering a riddle, running the first build and setting everything up upon receiving the source code, can be challenging to say the least. One little dependency version number can make or break a project. I’ve worked on so many projects where just a version mismatch can cause absolute chaos. Even give a false positive when the project successfully builds, but then is covered in runtime crashes and warnings. To save you the headache of countless video calls and messages about the correct procedure to build and run a project, just save yourself the time and provide the minimum instructions and notes required to fulfill that task.</p><p>Container tools such as<a href="https://www.docker.com/"> Docker</a> provide a great way for drafting a build contract that can be reused just for cases like this. It eliminates the need to disclose concrete version names and codes about the running platforms and dependencies (we will come back to this a bit later).</p><p>One important thing to mention here is to actually <em>test </em>the guide or container component you are providing on your own, local machine and some other test machine to ensure that the guide is not missing some crucial key or step.</p><h3>Project documentation</h3><p>For some reason, this miraculous and crucial piece of the puzzle, tends elude or even evaporate into thin air whenever the reaching hands of another developer come near it or even mention’s it.</p><p>The least amount of data contained in the documentation should be: what’s the project about, what was the state of the project when you started working on it (if it was a legacy project), what’s the project state at the moment of the handover with even mentioning potential unfinished tasks or goals and finally, what was the original plan for the project to end up to.</p><p>If there isn’t any documentation, write some. At least some description and the state the project’s currently in. The receiving party should have the least amount knowledge if the project is actually doing something or if it can be scrapped the moment the handover process is complete.</p><p>If there is existing project documentation, go over it once more. Check is everything is there and fill in if something you think might be important is missing.</p><h3>Account credentials</h3><p>Any assigned testing account credentials or parameters should be provided to ensure that the project can be explored by the receiving party as soon as possible.</p><p>There’s a possibility that some full account credentials for some sensitive services needed to be provided. Be sure to check with a supervisor or, if possible, the security team on how the sensitive data should be provided.</p><p>If you have shared accounts with other projects, decouple them before the handover. Create new accounts and optionally tailor them to the receiving party.</p><p>Check your Git history logs for any mentions or exposures for the sensitive data.</p><h4>Third party service accounts</h4><p>Clean up any required third party service accounts of any debug data and artifacts. Simply provide a clean slate with setting the last logs to be the snapshot state.</p><p>If cloud services were used during the development and production process, when providing the project, remove any traces of previous logs and data. Some services might not provide this feature, in which case you might consider creating new accounts and migrate to them before the handover process. Mention this to the other party, maybe they have their own existing accounts that they can use or even create new ones that they can share with you in order to update the project.</p><p>Be sure that you comply to any GDPR regulations when there’s mention of User data. If possible, check with someone who’s more familiar with the legal procedures for this topic.</p><h3>API documentation</h3><p>One would imagine that if a project contains or targets an API service, that some documentation or mention of all the possible calls and endpoints exist, but here we are, no documentation just mysterious calls to some outside source of information. Time to lift up the sleeves and start writing down a rudimentary API document yourself.</p><p>This might seem to be a lot of unnecessary work, especially for someone that perhaps wasn’t actively engaged in the API development process. Don’t worry, that’s why we are here. There are helpful tools that make generating API documentation a bit easier.<a href="https://www.postman.com/"> Postman</a> can be a good tool to start with or search online for alternatives. If using Postman, be sure to also share the API<a href="https://learning.postman.com/docs/getting-started/creating-the-first-collection/"> collection</a> and<a href="https://learning.postman.com/docs/sending-requests/managing-environments/"> environment</a>.</p><p>Try to print the network logs and see what data you can extract from that and go from there. Be sure to write down also any error states and network responses.</p><p>The minimum data to be written around a single API call should be:</p><ul><li>URL</li><li>Call method</li><li>Request data with potential query or path parameters</li><li>Response data</li></ul><h3>Code clean-up</h3><p>Just to be clear, you don’t need to rush and complete or fix the code. Nothing like that. What you should do is clear sensitive comments and debug messages. Any trivial references of yourself and/or your company should be checked and double checked. Needles to say, remove any course words and rude/silly comments or property names. Leave only the comments that provide value and information about the code itself.</p><p>Running a a standardized code formatter on the whole code base, ensures a bit of uniformity in the way the code is presented.</p><p>Refactor any file, class, method, field and property names to be more reflective of the functionality they provide. Run a global search on the whole project for keywords you deem to be important to remove before the handover.</p><h3>Source code</h3><p>Directly tied to the previous segment, source code maybe the most crucial part in the handover process. Chances are that the receiving party is primarily interested in the source code, but as we are seeing in this document this key component can be shadowed by so many missing or incomplete pieces that surround it. Many<a href="https://www.jetbrains.com/help/idea/import-project-or-module-wizard.html"> IDE’s</a> provide a way to export the source code for a project that makes it more easy to unpack and run. Export them and import them again under a different name, just to mimic how someone new to the project might experience that process.</p><p>If parts or the whole code itself should be copyrighted, make sure that it actually is. Check the wordings and date values. Any proprietary library or module, or something else that is subject to different copyright or licensing, be sure to disclose this to the receiving party.</p><p>Binaries or working examples and artifacts are a welcomed inclusion as well.</p><h3>Git clean-up</h3><p>Oh <a href="https://git-scm.com/">Git</a>, where would we be without the safety harness of this tool. Git knowledge is almost an art form and the more one knows how to use it, the more rewarding it becomes.</p><p>Git or any other VCS for that matter, provide a unique snapshot look into the various stages of development. All the different branches, commit messages and merge requests aiming towards one single outcome, a better project overall.</p><p>The more mature and complicated a project is, the more information is stored in the Git<a href="https://git-scm.com/docs/git-log"> Log</a>. There’s quite the possibility of old, temporary or even deprecated branches just sitting there without ever being merged into another branch or simply being marked for removal but never actually meeting that task. Go over the Git tree and make sure that only the necessary branches are left. I will not go into details about merging or re-basing branches, there are a lot more great resources available<a href="https://betterprogramming.pub/differences-between-git-merge-and-rebase-and-why-you-should-care-ae41d96237b6"> online</a> about that topic, but simply put re-basing ends up with a single progression line and is perhaps a bit easier to understand and merge shows the paths each of the merging branches have made.</p><p>If the Git history contains a lot of sensitive information, one simple thing to do is to just create a new repository from the latest branch and provide that as is. If questioned about the lack of log information, just say it’s outside your pay grade to discuss such things.</p><h3>Assets</h3><p>Assets can be any document,note, graphical element, fonts, translations, etc. that aren’t necessarily thought of as being an essential part of documentation, but nonetheless can provide a lot of value to someone just starting out with the project. For example, it can be very useful if we provide the Photoshop templates or settings we have used for the graphical elements in our project.</p><p>UX Design sketches, color codes, testing strategies and schemes can speed up a lot of tedious tasks around matching the exact specifications provided in some other document. I’m not saying that you should provide absolutely <em>every </em>single little brainstorming doodle, just the assets that might be forgotten in the hectic of handing over a project.</p><h3>Single repository</h3><p>I have yet to see a developer that likes when someone shares to them a bunch of links and scattered files and documents all over the place. Create a single asset repository which you can easily link and share to anyone that needs it. By using a single repository you’re creating a single source of truth for the files contained in it. Any writing/editing of the files contained in the repository should be disabled before sharing it with outside sources.</p><p>An additional step would be in defining a clear window of time when this repository is available to be read/downloaded. No repository should indefinitely be left open to any outside party. After the time window has passed, an internal backup copy should be generated and the repository deleted.</p><h3>Personal transition guidance</h3><p>Clearly state in a document and if possible in verbal conversation your availability for providing personal guidance or tutoring to new members of the development team or not. If you are available, make sure to state your time availability and window for when you can be contacted. Be very clear about this, you don’t need early morning calls about a project you’re no longer working on. The time window depends on the size of the project (the bigger the project, the larger the time window), but it should almost never exceed a month from the handover date. After that, it’s just up to your personal feelings if you’re going to be available for discussions about the project or not.</p><p>There’s also the possibility of you providing paid support, but this can heavily depend on the work contract you’ve signed with the company you were developing the project for or if you’re a freelance contractor.</p><h3>Snapshot version names (optional)</h3><p>While we have already mentioned containers and dependency versioning, it’s a good practice to write out also the version names and code of the tools and platforms that you have been working with on your local machine. For example: Visual Studio Code version, extensions used, Linux distro version (if developing on Linux systems), Git version and so on.</p><h3>Helpful tips (optional)</h3><p>Like any other optional step, its nice to go a bit beyond and show dedication to a project by sharing some helpful tips. I would usually recommend with sharing tips that you yourself wished you had someone tell you before joining the project. Almost all projects come with their own special “quirks” that are usually omitted when disclosing details. Sometimes a project has some unpredictable states and could cause confusion to the new developers. Write them down and be sure to mention that document to the receiving party, because a lot of developers tend to overlook documents that are not integral to the whole project (eg. API documentation).</p><h3>Solo developer transition</h3><p>So what to do when the receiving party is, well You, or better yet, there is no receiving party? Everything mentioned up to this point still stands. If your pausing a project for an unspecified amount of time, make sure that when the time comes for you to jump back in, you’ll do it with ease. So many developers can confirm the notion that they have a bit of a “snag” when they come back to a project after more than 6 months have passed. The truth is that many of them have evolved and transitioned over that period and different approaches and new experiences were made that the older versions of themselves simply didn’t know back then. So, when they open the old project with “fresh” eyes, all sorts of things tend to “pop” and even disorient. That’s when looking at some documentation can really help out.</p><p>Just as a side note, if they are disoriented by the very code they have written themselves, how do you think a complete newcomer would feel after looking at that code without any guidance?</p><h3>Sunsetting a project</h3><p>It’s always something heartbreaking when we have to simply put a project aside indefinitely. Depending on the passion, drive and effort put into a project up to that point, it can be very upsetting and even sad. There’s also a possibility that a project was just sitting and taking up storage space. So that’s why good preparations have to be made and much of the same steps I’ve mentioned before repeated.</p><p>If you have the sunsetting project listed somewhere (portfolio perhaps), be sure to update its status with a final note. Check for any links leading to the project and remove them. If the project was hosted on a third party service, look for a the option of first closing the project and then finally removing it. Export the source code files, assets, build scripts,zip them up and archive them.</p><p>There’s solid digital asset organization system called the<a href="https://fortelabs.co/blog/para/"> P.A.R.A.</a> method. The para stands for Project, Areas, Resources and Archive. Its a great way for putting every project into perspective and defining, for example common attributes for different projects. If following this method, when sunsetting a project, one just needs to move the project from the ‘Projects’ category to the ‘Archive’ one and that would be it with that.</p><p>I wouldn’t recommend outright deleting the repositories and files regarding a project. Although chances are very slim in revising that project again for some reason, one can never know. Maybe there’s just that one asset that is needed one more.</p><h3>Bad examples</h3><p>We have seen it in the industry happen all the time. Companies buying other companies and later learning about the issues with a product. Developers and other employees quitting when the start product changes hands.</p><p>Being possessive around code is very common and it’s a factor that should always be calculated in when transitioning personnel around a project (not just developers, the whole team).</p><p>Painting a wrongful picture of the current state of the project and “masking out” some problematic aspects of it, can cause devastating effects for the new team members. “Do a code review or demo before the handover.”- you might suggest, but here’s the thing, large projects have the “luxury” of being very difficult to analyze, by the sheer prospect of the time it takes to look over every single detail. The buyer, for example, is only interested in a single functionality of the buying product and doesn’t care about everything surrounding that single feature. This leads to acquisitions being made beforehand with the teams being among the last to be given the notice of it. Needless to say, the time and enthusiasm to prepare everything for the handover is very low at that point.</p><p>If you ever heard the answer: ”The code is the documentation”, on your inquiry about any missing documentation, you rightfully should be displeased. The documentation <em>is</em> the documentation, everything else is extra. Sadly, this is a very common scenario and we should aim to reduce it as much as possible at least by setting an example for ourselves. Since you’re already here, reading this article, you’ve made the first step to being better professionals than the person giving the answer mentioned at the beginning of this paragraph. Good job!</p><h3>Final notes</h3><p>Do yourself and others a huge favor and take the extra mile and ensure that you have done everything in your power to make a transition process go as planned.</p><p>Do you have any examples of good and bad project handovers? I’m looking forward to hearing about them and also about your thoughts on the subject matter. Be sure to comment and to share this post if you find it interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=caec9673b02b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>