<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Ministry of Programming — Technology - Medium]]></title>
        <description><![CDATA[Essays on Software Development in the Startup Trenches - Medium]]></description>
        <link>https://medium.com/mop-developers?source=rss----5d6f05ead37f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Ministry of Programming — Technology - Medium</title>
            <link>https://medium.com/mop-developers?source=rss----5d6f05ead37f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 12:35:51 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/mop-developers" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How Information Overload is Making Coders Compile Coffee Instead of Code]]></title>
            <link>https://medium.com/mop-developers/how-information-overload-is-making-coders-compile-coffee-instead-of-code-e52b12e0a050?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/e52b12e0a050</guid>
            <category><![CDATA[notifications]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[stress-management]]></category>
            <category><![CDATA[time-managment-skill]]></category>
            <dc:creator><![CDATA[Amar Bugarin]]></dc:creator>
            <pubDate>Wed, 14 Aug 2024 09:15:23 GMT</pubDate>
            <atom:updated>2024-08-14T09:15:22.975Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8z9eJymQ1DhSbb4Y.jpg" /><figcaption>Generated with Dalle-3</figcaption></figure><p>Hey there! Ever feel like your brain is a Dark Souls stamina bar, where every unnecessary bit of information you scroll through, read, and context switch drains your mental energy?</p><p>Well, you’re not alone. In my early career days, my biggest issue was managing my stamina bar. Instantly opening notifications on Slack or messages, trying to respond to everyone, and enduring long, profuse meetings left me exhausted.</p><p>There was no rhyme or reason I did not have a structure that would help me. I just jumped into tasks without planning, whether it was a new feature or someone needing my help. Consequently, my stamina bar dropped from 100 percent to <a href="https://www.youtube.com/watch?v=dQw4w9WgXcQ">what the hell is happening percent</a>, which isn’t enough to tackle big brain moves.</p><p>After experimenting, reading, and learning, I have found some ways to <strong>MIN/MAX</strong> my stamina, which has made me more productive and better at work. It also helps with my well-being.</p><h3>The Issue with Information Overload</h3><p>To begin with, our tech and tools have come a long way. Now, with ease, I can communicate with my client or friend from a different country or continent, keeping us connected in real-time with milliseconds of delay. You will always be informed and always notified; you are, in every sense of the term, connected.</p><p>However, tools like Slack and Gmail also have downsides, especially in a remote environment. They can easily sidetrack us from the main task. For example, when we are in deep work trying to debug something and get a Slack notification about random info or someone asking for help, it disrupts our focus.</p><p>I cannot emphasise this enough, but for the majority, you complete your best work when you’re in <strong>DEEP FOCUS</strong>. For every interruption, your brain needs at <a href="https://www.themuse.com/advice/this-is-nuts-it-takes-nearly-30-minutes-to-refocus-after-you-get-distracted">least 30 minutes</a> to refocus on the task at hand. This can be a significant productivity killer.</p><p>I hope this diagram can show you the insanity of trying to work like this, for every interruption you have about 20–30 min to refocus. OH and If you suffer from OCD apologies my mother did not raise a surgeon, that is for sure so the diagram is not that sharp there I say.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*dj7MLrANTlPa4AOMV2eM7w.png" /></figure><h3>Confession Time: Is Your Always On Mode Worth the Buzz?</h3><p>While being constantly online can sometimes be beneficial, especially for teamwork, it’s crucial to understand that this isn’t a one-size-fits-all approach, <strong>context is king</strong>. The effectiveness of always being available depends on your role and the flexibility of your team. It’s important to clarify that being highly responsive can indeed be advantageous, particularly when it’s a strategic choice that helps you stand out.</p><blockquote>However, for junior developers who remain ‘<strong>always on</strong>’ without a specific purpose or out of habit, this practice can lead to unnecessary depletion of stamina and health. And it can impact the work development output…</blockquote><h3>How to compile more</h3><h4>Streamlining Notifications</h4><p>In the early days, my devices were buzzing() non-stop. It’s common to find a variety of channels within an organization’s Slack, ranging from crucial updates to more social and informal interactions. If you’re being interrupted by notifications from less essential channels, here’s a straightforward solution:</p><ul><li><strong>Mute the Noise</strong>: If it’s just a casual chat that often pings your Slack, consider turning off notifications for those specific channels. This simple change can significantly reduce unnecessary distractions.</li><li><strong>Turn off non-essential alerts</strong>: Keep crucial notifications like family messages through iMessage or WhatsApp. For apps like Instagram and Twitter, disable notifications.</li><li><strong>Team Oriented Deep Work:</strong> Communicate with your team about when you will usually respond slower due to the task at hand, and when you will be free to interact quickly. I usually add a Focus status on Slack so that the team working with me understands that I will respond slower than usual.</li></ul><h4>Prioritise Your Tasks</h4><ul><li><strong>Eisenhower Matrix</strong>: Use this tool to prioritise tasks by urgency and importance. This helps in focusing on what truly matters, reducing the stress of handling everything at once.</li><li>The Pomodoro Technique: Work in focused bursts of 25 minutes followed by a 5-minute break. This can enhance concentration and prevent burnout from prolonged periods of work.</li></ul><blockquote>For me it was not useful but many of my colleagues use and swear by it, so try it out.</blockquote><h4>Leverage Technology Smartly</h4><p><a href="https://support.google.com/a/users/answer/9282734?hl=en"><strong>Automated Filters:</strong></a><strong> </strong>Set up rules in your email and Slack to automatically sort messages into folders or tabs based on their priority. This can help in managing the influx of notifications efficiently.</p><p><strong>Do Not Disturb</strong>: Activate ‘Do Not Disturb’ modes during critical work hours to silence non-essential notifications across all your devices.</p><h4>Integrate Well Being Practices</h4><p>Mindful Breaks: Incorporate short mindfulness exercises or stretching breaks during your workday to refresh your mind and body. Regular Check-ins: Set a weekly review routine to assess your productivity and well-being levels, adjusting your strategies as necessary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*ZbYtucDzT7mue26Y" /><figcaption><a href="https://www.reddit.com/r/Machinists/comments/16pjw5l/how_bad_is_your_desk_or_work_area_this_is_my_desk/">https://www.reddit.com/r/Machinists/comments/16pjw5l/how_bad_is_your_desk_or_work_area_this_is_my_desk/</a></figcaption></figure><h4>Don’t be a stereotypical messy developer</h4><p>We all know it’s tough to tidy up, especially when you’re serenaded by soothing notification sounds like this notification chime(). But really, a clean desk can transform your work vibe! Remember the saying: “A clean desk means a clear mind.” And no, “messy” doesn’t have to mean a mountain of fast food wrappers it’s about minimising distractions. Just stick to the essentials: a notebook, your laptop, and necessary peripherals.</p><p>Sure, some folks like Linus the maker of Linux might thrive in chaos, but let’s be honest, that’s pretty rare. Keep it simple…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e52b12e0a050" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/how-information-overload-is-making-coders-compile-coffee-instead-of-code-e52b12e0a050">How Information Overload is Making Coders Compile Coffee Instead of Code</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GitOps: A Brief Introduction]]></title>
            <link>https://medium.com/mop-developers/gitops-a-brief-introduction-7537fcef4d60?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/7537fcef4d60</guid>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[gitops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Bojan Raic]]></dc:creator>
            <pubDate>Mon, 20 May 2024 08:39:07 GMT</pubDate>
            <atom:updated>2024-05-20T08:39:07.623Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Have you ever fixed a random misconfiguration issue in production? Let me guess: it was super urgent, so you issued some commands to make things work as quickly as possible. Only for it to up again on re-deploy because you forgot to persist the fix? Fear not, you’re not alone! There is a method that streamlines deployments, boosts collaboration and cuts down on errors, and it does so by moving away from an imperative approach to making changes, to a declarative one: <strong>GitOps</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*gsR37RWSwP6mlHXgR-yp_Q.jpeg" /><figcaption><strong>Imperative vs. Declarative</strong></figcaption></figure><p>In this article, we will look at what GitOps is as a practice and define its core concepts. We will explore how GitOps enhances development lifecycle, and how each of the core principles of GitOps maps to different stages of the GitOps Flow. Finally, we will consider both the benefits and common challenges of implementing GitOps. By the end of this article, readers should be able to both understand the concepts and also start evaluating how well GitOps fits within their team or organization.</p><h3>GitOps Defined</h3><p>GitOps is a modern approach to continuous delivery best suitable for cloud-native environments built around Kubernetes. It leverages Git repositories (the hint was in the name, right?) to define, store and manage infrastructure and application definitions and configurations. As such, it is applicable to all major aspects of a typical software delivery ecosystem. From Infrastructure-as-code (IaC), configuration management, CI/CD pipelines, to actual application definitions and manifests, GitOps can be viewed as a universal, structured way of reasoning about systems. Once we have a clear mechanism for describing all aspects of our systems, we can treat (nearly) everything as code, and store the desired state as a collection of files within a Git repository. This notion is often referred to as <strong>“Everything-as-Code”</strong>, and is at the core of GitOps.</p><h3>GitOps Principles</h3><p>As the name suggests, GitOps is all about the automation of operational tasks and processes based on the declarative state stored in Git.<br>There are four key GitOps principles. Two focus on utilizing Git for declarative state management, while the other two center around automating operational tasks and processes based on that state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/674/1*CVvBryQ9z1NAOZWc6JJLoQ.png" /><figcaption><strong>Key Aspects of GitOps</strong></figcaption></figure><p>Let us define and discuss the core GitOps principles. A team or organization looking to implement GitOps should focus on the following four pillars:</p><ul><li>Declarative Configuration</li><li>Versioning and Immutability</li><li>Desired State Automation</li><li>Continuous Reconciliation</li></ul><p><strong>Declarative Configuration</strong> mandates defining the system’s desired state via files stored in one or more Git repositories. This includes infrastructure definitions, configurations, application code, and deployment manifests. Declarative configuration replaces the imperative model of issuing commands to define and update our system, ensuring consistency and reproducibility.</p><p>Using Git for storing the desired state naturally leads to the second pillar of GitOps: <strong>Versioning and Immutability</strong>. Git’s version control capabilities capture changes for <strong>historical and audit trail purposes</strong>.<strong> </strong>Git also facilitates reviewing and approval of changes to the desired state. Similarly, Git can also be used to <strong><em>revert</em></strong> any erroneous or unwanted changes introduced to the state of our system.</p><p><em>Git repositories act as </em><strong><em>the single source of truth</em></strong><em> about our system</em>, enhancing transparency and accountability. Git-driven controls and automations ensure consistent changes, thus reducing errors and time-consuming manual interventions.</p><p>The real strength of using Git repositories as the single source of truth is in automating the propagation of changes to target environments. This introduces the third pillar of GitOps: <strong>Desired State Automation</strong>. Since GitOps is a natural fit for cloud-native environment setups on top of Kubernetes, we can use Kubernetes itself to automatically apply the desired state changes. To do this, we utilize specialized controllers that act as GitOps agents watching the GitOps repository and comparing the desired state in the repository with the actual state of the cluster. Some of the most popular open-source GitOps agents include <a href="https://argo-cd.readthedocs.io/en/stable/">ArgoCD</a> and <a href="https://fluxcd.io/">Flux</a>.</p><p>GitOps agents not only automate application of desired state, but also continuously reconcile desired and actual states. Through a Continuous Reconciliation loop, these agents compare repository changes with the cluster’s state. This reconciliation process can be further enhanced with web hooks that can notify the agents of GitOps repository changes almost immediately. This makes reconciliation more responsive and minimizes the amount of time the cluster is in a potentially unwanted or drifted state.</p><p>Continuous reconciliation ensures automatic undoing of changes introduced via imperative commands, thus promoting <strong><em>immutability</em></strong> and enforcing structured change management through <strong><em>Git review and approval processes</em></strong>. In case of errors introduced manually or imperatively, the reconciliation loop acts as a <strong><em>self-healing mechanism</em></strong>, restoring the cluster to the desired, working state.</p><h3>GitOps Workflow</h3><p>With the theory behind the GitOps principles out of the way, let us now delve into how their implementation enhances a typical <em>Develop → Build + Test → Deploy</em> lifecycle. The relationship between GitOps principles and the GitOps workflow is illustrated below.</p><figure><img alt="GitOps: From Principles to Practice" src="https://cdn-images-1.medium.com/max/985/1*ZoNEDK3GaKr4faB7nI2esA.png" /><figcaption><strong>GitOps Principles ← → GitOps Workflow</strong></figcaption></figure><p>Details worth pointing out include:</p><ul><li>Commits to application repositories trigger an automated build + test CI/CD pipeline. Upon successful test execution, a container image is built, tagged and pushed to the container registry.</li><li>CI/CD system <strong><em>does</em></strong> <strong><em>NOT</em></strong> imperatively deploy the built application, but rather <strong><em>commits a configuration change to the GitOps repository</em></strong>, updating the image tag in the deployment manifest with the one that was just built.</li><li>GitOps Agent is configured to watch for changes within the GitOps repository and apply them automatically. It will detect that the desired state in the GitOps repository has changed and no longer matches the actual state within the cluster, since the running image tag does not match the desired (newly built) image tag.</li><li>GitOps Agent then figures out what all needs to be changed (in this case, just the image tag reference of the application deployment, and then applies the changes to the underlying Kubernetes resources.</li><li>Kubernetes will pull the specified image the from the container registry and replace the running instance of the application with a new version.</li><li>GitOps Agent will continuously monitor for drift between the actual and desired state of the cluster and run the reconciliation loop to ensure the state of the cluster matches the state declared within the GitOps repository.</li></ul><p>A similar flow kicks in when the GitOps repository is directly updated by committing changes to other configuration-as-code or infrastructure-as-code files. Pipeline implementation will differ, of course, but the changes can still be applied automatically by an agent, running either remotely or within the cluster.</p><p>While the GitOps agent commonly runs inside the Kubernetes cluster it operates on, it is also possible to configure a single agent to communicate with remote clusters. Such setup allows for centralized management and orchestration of changes across multiple clusters and environments.</p><h3>GitOps Benefits</h3><p>Implementing a solid GitOps practice has many benefits for the entire organization, albeit most notably for dev, ops, and platform teams. While many were either mentioned or hinted at while discussing individual GitOps principles, here are some additional benefits worth highlighting:</p><ul><li><strong>Familiarity</strong>: Git and related flows and processes are familiar to most technical personnel, regardless of the technology stack in question</li><li><strong>Technology stack agnostic</strong>: if it can run on Kubernetes and be expressed declaratively, GitOps can be used. Whether we’re dealing with microservices or monoliths, Java, Ruby or Python, Pulumi, Terraform or something else entirely — GitOps setups are inherently agnostic to the technology stack of choice.</li><li><strong>Vendor-neutral</strong>: because Git, a very popular open-source version control system, is at the core of GitOps, GitOps setups are also often vendor-neutral and flexible. If at some point we wish to move from BitBucket to GitLab, for example, the migration path is often much more straightforward with GitOps than with an imperative-driven setup, where we would need to worry about vendor-specific configurations, pipeline syntax and more.</li><li><strong>Support for multiple Kubernetes-oriented tools</strong>: GitOps agents typically support more than one Kubernetes-oriented tool; for example, in addition to kubectl, one can easily utilize Helm, Kustomize, Jsonnet, or even make use of custom tooling, like SOPS or Hashicorp Vault, all in a GitOps-friendly manner.</li><li><strong>Improved security and reliability</strong>: In an imperative approach, CI/CD system often has direct access to the underlying cluster or cloud environment, which can be problematic from a security standpoint. With GitOps, an agent typically runs within the cluster and thus the risk of exposure is smaller. At the same time, because all changes are reviewed and approved before actual rollout, risk of mistakes is often reduced. As mentioned before, undoing mistakes can be done as simply and as quickly as reverting a problematic Git commit. All of this helps improve system reliability over time.</li><li><strong>Enhanced developer experience, speed &amp; frequency of deployments</strong>: because GitOps relies on automation at every possible step in the lifecycle, it often gives way to noticeable productivity gains, leading to more frequent deployments and better developer experience overall.</li><li><strong>Increased visibility &amp; collaboration</strong>: storing nearly everything in Git increases visibility, which in turn empowers team members to better comprehend the system, suggest improvements through pull requests and reviews, and scrutinize the overall system more thoroughly than they could otherwise.</li></ul><h3>GitOps Challenges</h3><p>Like any technology-related concept, GitOps also comes with its own set of challenges. Here are some drawbacks and challenges worth noting:</p><ul><li><strong>Structuring GitOps repo(s)</strong>: storing everything in a single repository, while potentially appealing at first, is almost always a bad idea. On the other hand, being too granular can lead to too many GitOps repos at play. Additionally, planning repository structure for multiple environments, clusters, regions or cloud providers can be difficult and requires careful considerations</li><li><strong>Git conflicts</strong>: Automating nearly everything can sometimes lead to Git conflicts. When multiple pipelines write to the same repository simultaneously, Git conflicts can disrupt the whole process. In such scenarios, pipelines or other mechanisms need to be able to either avoid or resolve conflicts seamlessly.</li><li><strong>Secrets management</strong>: Managing secrets can be a significant challenge with the GitOps approach, since it mandates storing everything, including confidential information, as files in Git. This in turn requires encrypting sensitive files before storing them in Git. Consequently, the chosen GitOps agent must be able to decrypt these files dynamically in order to create Kubernetes secrets efficiently.</li><li><strong>Complexity at scale</strong>: In addition to structure of the GitOps repositories, scaling out GitOps practices to multiple environments, clusters, regions and/or cloud providers introduces a unique set of challenges from an orchestration point of view</li><li><strong>Cost</strong>: An overly complex GitOps setup, whether due to excessive tooling, infrastructure, or maintenance overhead, can result in increases in costs over time. While this is typically not directly related to GitOps in and of itself, it’s crucial to note that adopting GitOps practices does not inherently shield against cost escalation.</li></ul><h3>Conclusion</h3><p>GitOps is a natural fit for cloud-native, Kubernetes-oriented environments. It offers many benefits over other alternatives, such as advanced degrees of automation, drift detection and reconciliation, easy audit trail and history of changes, quicker rollbacks, enforcement of code reviews and approvals, and faster and more frequent deployments. While not without its challenges, a carefully considered and continually improving GitOps practice has a lot to offer to both technical and business teams within any organization embracing cloud-native and Kubernetes.</p><h3>Useful Links</h3><p>Interested in learning more about GitOps and related concepts? Check out these links to explore more:</p><ul><li><a href="https://opengitops.dev/">OpenGitOps</a></li><li><a href="https://www.gitops.tech/">GitOps Book</a></li><li><a href="https://argo-cd.readthedocs.io/en/stable/">ArgoCD</a></li><li><a href="https://fluxcd.io/">Flux</a></li><li><a href="https://helm.sh/">Helm</a></li><li><a href="https://kustomize.io/">Kustomize</a></li><li><a href="https://getsops.io/">SOPS</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7537fcef4d60" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/gitops-a-brief-introduction-7537fcef4d60">GitOps: A Brief Introduction</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How JavaScript Closures Can Improve Your State?]]></title>
            <link>https://medium.com/mop-developers/how-javascript-closures-can-improve-your-state-5e84521790bd?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/5e84521790bd</guid>
            <category><![CDATA[javascript-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[javascript-tips]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Samed Skulj]]></dc:creator>
            <pubDate>Fri, 10 May 2024 08:41:08 GMT</pubDate>
            <atom:updated>2024-05-10T08:41:08.682Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="How JavaScript Closures Can Improve Your Scope?" src="https://cdn-images-1.medium.com/max/1024/1*SvDAgZRMa6wyyxbRAsckJg.png" /></figure><p>JavaScript is one of the most popular programming languages in the world, widely used for both client-side and server-side applications. One of its most powerful features, often not fully understood by new developers, is the concept of closures. Closures are a fundamental aspect of JavaScript, enabling powerful programming techniques and helping manage scope in original ways. In this blog, I’ll explore what closures are, how they work, and why they are so useful, providing examples to illustrate these concepts clearly.</p><h3>What is a Closure?</h3><p>At its core, a closure is the combination of a function and the “lexical” environment within which that function was declared. This environment consists of any local variables that were in-scope at the time the closure was created. Essentially, <strong>closures allow a function to access those variables from an outer function even after the outer function has finished execution</strong>, which leads to various useful applications.</p><h4>Lexical Scoping Explained</h4><p>To understand closures, you first need to understand lexical scoping. JavaScript uses lexical scoping, meaning that a function’s ability to access variables is determined by its location within the source code. Let’s consider a simple example to see how lexical scoping works:</p><pre>let scope = &quot;global scope&quot;; // A global variable<br>function checkScope() {<br>  let scope = &quot;local scope&quot;; // A local variable<br>  function f() {<br>    return scope;<br>  }<br>}<br><br>checkScope(); // Returns &quot;local scope&quot;</pre><p>In this example, <strong><em>checkscope()</em></strong> defines a local variable <strong><em>scope</em></strong> and a function <strong><em>f()</em></strong> that returns the value of the local variable. The function <strong><em>f()</em></strong> is executed within the same scope it was defined, which is why it <strong>returns “local scope.”</strong></p><h4>How Closures Work</h4><p>Now, let’s modify the previous example slightly:</p><pre>let scope = “global scope”;<br>function checkscope() {<br>  let scope = “local scope”;<br>  function f() {<br>      return scope;<br>  }<br>  return f;<br>}<br><br>let s = checkscope();<br>s(); // What does this return?</pre><p>In this version, <strong><em>checkscope()</em></strong> returns the function <strong><em>f()</em></strong> itself, rather than the result of <strong><em>f()</em></strong>. When <strong><em>f()</em></strong> is called (<strong><em>from</em></strong> <strong><em>s() in our case</em></strong>), it still accesses the local <strong><em>scope</em></strong> variable from <strong><em>checkscope()</em></strong>. Despite being executed outside of its original scope, <strong><em>f()</em></strong> remembers the <strong><em>scope</em> </strong>variable. This capability to <strong>”remember”</strong> the environment in which it was created is the essence of a closure.</p><h4>Practical Uses of Closures</h4><p>Closures have numerous practical uses, such as:</p><ul><li><strong>Data Privacy</strong>: Closures can create private variables and methods. This is useful in many design patterns, like the module pattern, where we encapsulate some of the code to shield it from the global namespace and create private states.</li><li><strong>Event Handlers</strong>: They can maintain state in an event without needing global variables or complex object properties.</li><li><strong>Functional Programming</strong>: Closures facilitate functional programming patterns, such as currying and partial applications.</li></ul><h4>Advanced Example: Creating Private States</h4><p>Consider this advanced example where closures are used to create private states:</p><pre>function counterForApple() {<br>  let apple = 0;<br>  return {<br>    count: function () {<br>      return apple++;<br>    },<br>    reset: function () {<br>      apple = 0;<br>    },<br>  };<br>}<br>let firstApple = counterForApple();<br>let secondApple = counterforApple();<br><br>console.log(firstApple.count()); // =&gt; 0<br>console.log(secondApple.count()); // =&gt; 0: they count independently<br><br>firstApple.reset();<br><br>console.log(firstApple.count()); // =&gt; 0: because we reset firstApple<br>console.log(secondApple.count()); // =&gt; 1: secondApple was not reset</pre><p>In this case, each call to <strong><em>counterForApple()</em></strong> creates <strong>a new closure</strong> with its own <strong><em>apple</em></strong>. The <strong><em>count </em></strong>and <strong><em>reset</em></strong> methods are both closures, sharing access to the same <strong><em>apple</em></strong>.</p><p>Maybe some of you who are reading this, would compare this behaviour to <strong><em>OOP languages </em></strong>where we have private variables.</p><h3>Conclusion</h3><p>Closures are a powerful feature of JavaScript, providing both utility and efficiency in code management. They allow for elegant ways to handle data privacy, manage state, and write more maintainable and error-free code by keeping everything local to its execution context. Understanding closures is crucial for any JavaScript developer looking to increase the full potential of this versatile language.</p><p>By learning the concepts of closures and lexical scoping, developers can write cleaner, more efficient, and more secure JavaScript code, preparing the way for advanced programming techniques and applications. Whether you’re managing private states or handling complex application flows, closures are an important part of the JavaScript toolkit.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e84521790bd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/how-javascript-closures-can-improve-your-state-5e84521790bd">How JavaScript Closures Can Improve Your State?</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Forget the Hype: AI Isn’t Taking Your Coding Job]]></title>
            <link>https://medium.com/mop-developers/forget-the-hype-ai-isnt-taking-your-coding-job-9047f2d16171?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/9047f2d16171</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Faris Zacina]]></dc:creator>
            <pubDate>Wed, 17 Apr 2024 09:16:14 GMT</pubDate>
            <atom:updated>2024-04-17T09:44:02.874Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wuIafWdocHrzZrmmEZYjww.jpeg" /><figcaption>A computer with an absent programmer</figcaption></figure><p>A year ago, I argued that AI wouldn’t replace programmers. Guess what? I was right. AI is a tool, not a threat.</p><p>No programmers have been replaced by AI at my company, <a href="https://ministryofprogramming.com/">Ministry of Programming</a>, or in the companies we work with. AI continues to be used as a helpful assistant by hundreds of developers.</p><p>I’ll make my prediction once more:</p><blockquote>2024 won’t see programmers replaced — it’ll see a massive expansion in the overall workforce! AI is lowering the barriers to coding, fueling new businesses, and driving incredible innovation</blockquote><p>In 2024, programmers won’t just survive — they’ll thrive, and here is why AI is not the threat it seems to be.</p><h3>Conflating GAI with AGI</h3><p>Many people think generative AI will replace programmers because they confuse it with AGI (artificial general intelligence). The reality is, that today’s generative AI is like the most advanced autocomplete ever — a powerful assistant, not a conscious replacement for human programmers.</p><blockquote>Autocomplete on Steroids ≠ a Thinking Coder</blockquote><p>To truly replace programmers, we need AGI or even ASI (artificial superintelligence). Current tools, while impressive, still require human oversight and understanding to produce usable code.</p><h3>Hallucinations and the effect on speed and quality</h3><p>Even the best generative AI models struggle with what are called ‘hallucinations’ — seemingly correct code that’s flawed. Even an 1% hallucination/error rate can be a nightmare because software demands near perfection to function correctly.</p><p>Sure, AI copilots can boost a skilled programmer’s speed tremendously. But the moment a hallucination slips in, that speed gain evaporates. Suddenly, you’re debugging code you didn’t even write, which can be far slower than writing it yourself.</p><p>We’re naturally better at understanding our own code. Deciphering and debugging code written by someone else (or an AI) is a much tougher task. Even if the AI generates thousands of good lines of code, a single hidden bug can create days of frustrating work to fix it.</p><blockquote>LLM’s could generate super-bugs that take 10 or 100x more time to fix.</blockquote><p>Developers are very reluctant to read and interpret large amounts of code written by someone else. That will continue to be the case even with AI-generated code.</p><blockquote>AI assistant will speed you up, but they will also slow you down when you debug their hallucinations or suboptimal solutions.</blockquote><p>As Gary Marcus pointed out recently, one of the scariest problems with LLMs is that they will surely generate <a href="https://www.gitclear.com/coding_on_copilot_data_shows_ais_downward_pressure_on_code_quality?utm_source=substack&amp;utm_medium=email"><strong>lower-quality code</strong></a> and security issues.</p><p>We should always remember that even today most coders spend 90% of their time reviewing existing code and debugging, and not writing new code. How much will we spend reviewing AI-generated code that we don’t understand that well?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ltpj2_Ayot6NUHHQWJE5zg.png" /></figure><h3>Prompting is still programming</h3><p>Watch any programmer using AI copilots, and you’ll realize that writing good prompts is its form of programming. You need deep coding knowledge to effectively tell the machine what you want, understand the quality of its output, and assess if it’s good enough for real-world use.</p><p>While AI can generate decent code, it doesn’t think like we do. It can’t judge what’s “good enough” for a human user or guarantee production-level quality. That’s still a human skill. You can call it taste if you like.</p><blockquote><em>The secret to great code is iteration, not one-time generation.</em></blockquote><p>The best code comes from refining and improving what already exists. You need to understand the goal and the quality standard. AI tools often produce subpar solutions initially as they can’t interpret what is good enough and when to stop or iterate further, and that’s where human expertise and judgments come in.</p><p>Humans understand human requirements better than an autocomplete tool does.</p><h3><strong>Smaller Teams, Bigger Impact</strong></h3><p>While some companies might attempt to embrace smaller, AI-assisted development teams for increased productivity, others will get more ambitious and expand their teams as productivity increases — they can get more done, which speeds up growth.</p><p>Overall, even the temporary reductions in teams will not mean a shrinking job market for programmers. As one developer shifts roles, another opportunity opens somewhere else.</p><p>AI reduces the barrier to entry for founders to start new companies, so we might see a significant increase in the number of new companies created, which creates an even bigger market for software developers.</p><p>Ultimately, this fragmentation is positive. There will be many more smaller companies in the world.</p><blockquote>While larger companies may downsize their programming teams, those skilled AI-assisted developers will rapidly find opportunities with smaller, growing businesses boosted by the AI revolution.</blockquote><p>So even if some companies replace programmers because of AI productivity gains, those same people will quickly find a home in the next company that got more ambitious, or a smaller company that was just created and enabled by AI.</p><h3><strong>What about Devin, Magic.Dev and other AI agents replacing Devs?</strong></h3><p>Devin is cool, but fancy agents that can do small parts of a dev job are not replacing anyone, just augmenting people.</p><p>To not repeat what others have written, I will just quote Gergely Orosz here:</p><blockquote>So far, all we have is video demos, and accounts of those with access to this tool. From the videos, it seems it’s more of an advanced agent that can switch between certain tools. Still, at this point, Devin doesn’t feel more than a heavily work-in-progress prototype. — Gergely Orosz</blockquote><p>Even if Devin gets better, it will generate weird code, hard-to-fix bugs, and many other issues, so I don’t see it replacing a single human programmer this year.</p><iframe src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//twitter.com/findsakshatdev/status/1776493667433566224%3Fs%3D46&amp;image=" width="500" height="281" frameborder="0" scrolling="no"><a href="https://medium.com/media/c4350b79be57b941c02a6a8dea5655d4/href">https://medium.com/media/c4350b79be57b941c02a6a8dea5655d4/href</a></iframe><h3>Conclusions</h3><p>Forget the fearmongering — AI isn’t replacing programmers anytime soon. Think of these tools as incredible assistants, a major upgrade to your coding environment. It’s like autocomplete on steroids!</p><blockquote>The future of coding are enhanced human AI-assisted programmers, not obsolete ones.</blockquote><p>AI won’t make programmers extinct. Instead, programmers who embrace AI will become unstoppable. Barriers to coding will fall, unleashing a flood of new ideas and businesses. This is a golden age for software developers!</p><blockquote><strong>The good news is the claim “you will not need software engineers” has been on repeat since the 1960s. — Gergely Orosz</strong></blockquote><p>AI will make coding accessible to many more people and companies. And for seasoned programmers? It’s a productivity multiplier.</p><p>AI assists even those programmers who have been out of the game for a while. It accelerates their return to active coding.</p><p><strong>Some teams might get smaller, but overall, there will be much more programmers in the world and not less.</strong></p><p>Get ready to build amazing things, faster than ever before.</p><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged remote-optimized venture builder specialized in building startups and new products💡 We were voted in the top 1000 fastest-growing companies in Europe by the Financial Times.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups toward product success.</em></p><p>*If you want to work with us on your startup feel free to contact us at — <a href="https://ministryofprogramming.com/contact/*">https://ministryofprogramming.com/contact</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9047f2d16171" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/forget-the-hype-ai-isnt-taking-your-coding-job-9047f2d16171">Forget the Hype: AI Isn’t Taking Your Coding Job</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When Kubernetes and Go don’t work well together]]></title>
            <link>https://medium.com/mop-developers/when-kubernetes-and-go-dont-work-well-together-54533bb6466a?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/54533bb6466a</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Emin Laletovic]]></dc:creator>
            <pubDate>Fri, 12 Apr 2024 08:15:56 GMT</pubDate>
            <atom:updated>2024-04-15T14:02:59.373Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Go is not aware of the limits set for its container, causing some issues not easy to track. This is a story about how I stumbled into one of them.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*cG6_1bYuVomyjGQGO6BKfg.png" /><figcaption>AI prompt: Gopher breaking a container</figcaption></figure><p>In this article, I will cover a specific issue encountered in one of our production environments, go over the root cause, and a potential resolution. The title hints at the underlying problem, but we will cover the details of what happened.</p><h3>What happened?</h3><p>I got a bug report that there is a 502 Bad Gateway error happening when one of our API service endpoints, in a specific production environment, gets called. The calls for the same API endpoint are successful in all other production environments, just not this one.</p><p>Started with checking the usual stuff. Error logs, nothing. Tried to reproduce it, the issue is still there, but there aren’t any error logs or any other logs, at all. I noticed that the Kubernetes pod container gets restarted each time I call the API service endpoint. The first thought that came to mind was that the service panicked and the container restarted, so there must be logs in the previous pod instance. Checked there, nada. Next, I added more info logs to see where the issue happened exactly. It pointed to the issue happening when the json.Marshal function call was being executed, but there was nothing odd in the code. It was time to dig a bit more.</p><p>I shifted my focus to the Kubernetes pod and how it behaves when the issue happens. While the new pod is starting, I notice that the old pod has the OOMKilled status. OOMKilled? But that’s out-of-memory status when the container inside the pod goes over the defined RAM limit. What the hell is this API endpoint response trying to return?? To be fair, we built this endpoint to retrieve the bulk data for our mobile app so it can work in the offline mode. Even so, I didn’t expect that there would be that much data because these are the resource limits for the API service container:</p><pre># part of the deployment file<br>- name: api<br>  image: gcr.io/xyz/api:989d83a73a9d1605ac601e3ef0fdef5711dcffe4<br>  imagePullPolicy: Always<br>  ports:<br>  - containerPort: 5500<br>  resources:<br>    limits:<br>      cpu: 200m<br>      memory: 400Mi<br>    requests:<br>      cpu: 100m<br>      memory: 200Mi</pre><p>The RAM limit for the API service container is 400 MB. I doubled the RAM limits to see how far this issue goes. The deployment was reconfigured and a new pod with the new container limits was deployed. I triggered the API endpoint again and it returned the response successfully. The size of the response is 66 MB.</p><p>Hmm, the API service container with RAM limits of 400 MB should have been able to handle the 66 MB response with ease. How did it go out of memory then? After reading through some documentation and articles about Go internals (see the references at the end), some hunches about Go’s garbage collector started to form.</p><h3>As Monk would say, here’s what happened…</h3><p>Two things are important to explain how the issue manifests.</p><p>First, the Go’s garbage collector. I won’t go into all the details on how Go’s garbage collector works. I linked some great articles at the end if you want to read more about the details. One important detail for the issue at hand is that Go’s GC expands the heap memory when needed. At each Go’s GC run, it determines if more memory needs to be allocated, and then allocates it.</p><p>Next, this is the part where we go back to the title of this article. When the Go program (as a single process) runs in a container in a Kubernetes pod, by default it doesn’t know the resource limits set for the container it is running in. For example, let’s assume that our container is running on a multi-core machine with 8 GB of RAM. Assuming the heap continually rises, the Go runtime will eventually try to allocate all available memory of those 8 GB, even if other containers are deployed on that same machine, running other processes.</p><p>Combining these facts, we get how the issue manifests. While executing the json.Marshal function, the Go’s GC sees there is a need for more heap memory, and allocates it. It does that repeatedly in several GC cycles. When the process in the container goes over the memory limit (400 MB), the container’s system kernel terminates the process, with an out-of-memory error (OOM).</p><h3>Hunches are fine, but show me the facts</h3><p>Let&#39;s check GC trace logs to get some data on what happened. You can check these logs by setting the environment variable GODEBUG=gctrace=1. There are a few ways to do it, I added it to my Dockerfile.</p><pre>#The last part of the multi-step build<br># Build minimal image<br>FROM alpine<br>COPY --from=0 /go/bin/api /go/bin/api<br>ENV GODEBUG gctrace=1<br># Expose port<br>EXPOSE 5500<br># Run<br>ENTRYPOINT [ &quot;/go/bin/api&quot; ]</pre><p>Looking at the logs below, there is one more thing to note. There is another environment variable, GOGC, that determines how much the allocated heap will grow before the next GC run. GOGC is set by default to 100, which means it will grow by 100%.</p><pre>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;GetBulkDmsFiles: done&quot;,&quot;time&quot;:&quot;2024-02-25T16:06:08Z&quot;}<br>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;marshal response&quot;,&quot;time&quot;:&quot;2024-02-25T16:06:08Z&quot;}<br>gc 14 @247.533s 0%: 0.073+195+0.040 ms clock, 0.14+0/10/97+0.080 ms cpu, 138-&gt;138-&gt;117 MB, 138 MB goal, 0 MB stacks, 0 MB globals, 2 P<br>gc 15 @248.448s 0%: 0.061+280+0.041 ms clock, 0.12+0/89/2.9+0.082 ms cpu, 309-&gt;309-&gt;260 MB, 234 MB goal, 0 MB stacks, 0 MB globals, 2 P</pre><p>From the logs, we can see that the heap grows with each GC run. Please note that the last entry has 260 MB, which is what the Go program was able to write before it got terminated. In the next GC run, the heap would grow over 500 MB, more than the 400 MB limit defined in the deployment file for the API service container.</p><h3>Is there a solution for this?</h3><p>Everyone’s favorite answer, it depends. Some will prefer to leave this as is, to have a container that fails fast, not to tinker with how the GC works, and to deal with the functionality itself. This API endpoint that returns bulk data could have been rewritten to stream the data back, rather than to send one big response.</p><p>However, if it bothers you that your Go program and Kubernetes are kind of out of sync, there is a way to tune it to work more smoothly. The GOMEMLIMIT environment variable, introduced in Go 1.19, can be used as a soft memory limit for the GC. The OOM can still happen, but the GC will run more aggressively when it reaches the limit and it will make sure to use all of the live heap rather than keep increasing the heap beyond the limit.</p><p>In the API service’s deployment YAML file, I’ve added this environment configuration:</p><pre>env:<br>  - name: GOMEMLIMIT<br>    valueFrom:<br>      resourceFieldRef:<br>        resource: limits.memory</pre><p>The GOMEMLIMIT is set to whatever is defined as a memory limit in the deployment YAML file. This way, whenever I change the memory limits, the GOMEMLIMIT will be changed accordingly.</p><blockquote>It would be better to be able to define something like “90% of the memory limits”, but there is still no good way to do this, other than maybe using requests.memory. This <a href="https://github.com/kubernetes/kubernetes/issues/91514">issue</a> is still open. The <a href="https://github.com/KimMachineGun/automemlimit">automemlimit</a> library does that (and more, with cgroups), but it has to be included in the code.</blockquote><p>When I call the same endpoint, it returns the response successfully. Here are the GC logs and what the memory looks like before and after the call.</p><pre>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;GetBulkDmsFiles: done&quot;,&quot;time&quot;:&quot;2024-02-25T16:31:01Z&quot;}<br>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;marshal response&quot;,&quot;time&quot;:&quot;2024-02-25T16:31:01Z&quot;}<br>gc 18 @92.420s 0%: 0.053+190+0.097 ms clock, 0.10+0/98/95+0.19 ms cpu, 115-&gt;115-&gt;93 MB, 125 MB goal, 0 MB stacks, 0 MB globals, 2 P<br>gc 19 @93.019s 0%: 0.066+102+0.035 ms clock, 0.13+0/4.0/89+0.071 ms cpu, 189-&gt;189-&gt;165 MB, 186 MB goal, 0 MB stacks, 0 MB globals, 2 P<br>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;marshal response done&quot;,&quot;time&quot;:&quot;2024-02-25T16:31:02Z&quot;}<br>gc 20 @93.821s 0%: 0.066+193+0.004 ms clock, 0.13+0/2.6/3.6+0.009 ms cpu, 359-&gt;359-&gt;197 MB, 330 MB goal, 0 MB stacks, 0 MB globals, 2 P<br>{&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;response writer write done&quot;,&quot;time&quot;:&quot;2024-02-25T16:31:10Z&quot;}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xRBEB8ltD7z0JiINBS-_4A.png" /><figcaption>API service memory dashboard</figcaption></figure><h3>Final thoughts</h3><p>The presented solution is no silver bullet. There are scenarios where failing fast with the default settings is preferable to squeezing all the juice from the heap before the process is terminated with the OOM error.</p><p>That being said, the presented solution might help with better resource utilization, especially when multiple services are deployed on the same machine.</p><p>There are great articles about this topic already out there. I found out about them kinda late, it would have saved me a lot of time (by not writing this one haha). You should read it to get even more familiar with the topic. I’ve included them in the references below.</p><p>Please let me know if there are similar and more elegant solutions to tune your Kubernetes resources and Go programs, I would like to hear about it!</p><h3>References</h3><ul><li>Ardan Labs article about tuning the CPU limits (I got the idea from there) — <a href="https://www.ardanlabs.com/blog/2024/02/kubernetes-cpu-limits-go.html">https://www.ardanlabs.com/blog/2024/02/kubernetes-cpu-limits-go.html</a></li><li>A cool article about GOMEMLIMIT that has way more details — <a href="https://weaviate.io/blog/gomemlimit-a-game-changer-for-high-memory-applications">https://weaviate.io/blog/gomemlimit-a-game-changer-for-high-memory-applications</a></li><li>Ardan Labs article about the same topic — <a href="https://www.ardanlabs.com/blog/2024/02/kubernetes-memory-limits-go.html">https://www.ardanlabs.com/blog/2024/02/kubernetes-memory-limits-go.html</a></li><li>The article on the same topic — <a href="https://billglover.me/2022/09/14/use-the-kubernetes-downwards-api-to-set-gomemlimit/">https://billglover.me/2022/09/14/use-the-kubernetes-downwards-api-to-set-gomemlimit/</a></li><li>A guide to the Go’s garbage collector — <a href="https://tip.golang.org/doc/gc-guide">https://tip.golang.org/doc/gc-guide</a></li></ul><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged remote-optimized venture builder specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>If you want to work with us on your startup feel free to contact us at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54533bb6466a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/when-kubernetes-and-go-dont-work-well-together-54533bb6466a">When Kubernetes and Go don’t work well together</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using ClickHouse for Charts & Profits]]></title>
            <link>https://medium.com/mop-developers/using-clickhouse-for-charts-profits-ad6dc56abf67?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/ad6dc56abf67</guid>
            <category><![CDATA[trading]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[clickhouse]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Adis Nezirović]]></dc:creator>
            <pubDate>Thu, 04 Apr 2024 08:30:34 GMT</pubDate>
            <atom:updated>2024-05-04T15:16:25.455Z</atom:updated>
            <content:encoded><![CDATA[<h3>Using ClickHouse for Financial Charts</h3><p>How to stop worrying and embrace eventual consistency</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TWzFRtUBmqx46d_OqMYAvQ.jpeg" /></figure><h3>Intro</h3><p>While preparing this article, a book by the late professor Niklaus Wirth came to mind — “<em>Algorithms + Data Structures = Programs”. </em>While written in different times (50 years ago!), I think it still holds today.</p><p>The data consumed or produced by programs defines the architecture, operations and life-cycle of the application. It’s not enough to <em>know your algorithms</em>, to expand your engineering <em>skillz </em>you should leave the safe waters of conventional data backends (relational databases, plain old files) and embrace innovative and different systems like ClickHouse (or Kafka, Cassandra, S3). All of these have different tradeoffs, but also open up a world of possibilities for your app.</p><p>On the surface, <a href="https://en.wikipedia.org/wiki/ClickHouse">ClickHouse</a> is just a <a href="https://en.wikipedia.org/wiki/Column-oriented_DBMS">columnar</a> variant of a relational database system, but there’s more than meets the eye. From the very roots at Yandex, the defining property of ClickHouse is practicality, the features have been driven by the need of real companies and systems. It also helped that another Internet behemoth, <a href="https://www.cloudflare.com/">Cloudflare</a>, started using, improving and <a href="https://blog.cloudflare.com/http-analytics-for-6m-requests-per-second-using-clickhouse">blogging</a> about ClickHouse soon after it became freely available.</p><p>While it started as proprietary software, ClickHouse was eventually released to the world under permissive Apache license. I can’t emphasize this enough. If you can help it, keep the data in the open source databases, in open formats, no hostages allowed!</p><p>If you have the time, read some ClickHouse <a href="https://medium.com/mop-developers/reliable-apps-with-ha-proxy-part-5-7d7f269c5772">intro</a> in our <a href="https://medium.com/mop-developers/reliable-apps-with-ha-proxy-intro-bf48472c96bf">HAProxy series</a>.</p><h3>ClickHouse — TLDR;</h3><h4>Use ClickHouse …</h4><ul><li>if you have analytical, aggregation heavy data set (<a href="https://en.wikipedia.org/wiki/Online_analytical_processing">OLAP,</a> Pivot table)</li><li>if you don’t care about <em>persisting</em> individual rows one by one, but do care about <em>processing</em> all the rows</li><li>if a risk of having some data duplicates is acceptable</li><li>if your queries process large number of rows, but relatively small number of columns</li><li>if you want to process and save vast amount of data (petabytes!)</li><li>if your data source is a continuous stream of records/messages.</li></ul><h4>Don’t use ClickHouse …</h4><ul><li>if you can describe your data as “transactions” (<a href="https://en.wikipedia.org/wiki/Online_transaction_processing">OLTP</a>)</li><li>if you mutate a lot of data in place (SQL DELETE/UPDATE)</li><li>if you can’t live without referential integrity and normalized data</li><li>if you need database indices to optimize access to individual rows (or a small set of resulting rows).</li></ul><h3>Intro Quest — Charts</h3><p>Talk is cheap, let’s get our hands <em>dirty (and done cheap)</em> with sample data. Imagine we’re exploring a trading system (forex, shares, crypto, etc.). Before committing ourselves, we first want to learn from history and see if we can interpolate future trends (strictly speaking, past performance does not predict the future, but we have vivid imagination around here).</p><p>First, lets download some <a href="https://www.histdata.com/">historical data</a> for our playground. We’ll download forex data for <em>EURUSD</em>, 1min signals for the whole year 2023. In real life, your signal would be more frequent, you’d collect multiple events per second, but this will suffice for now.</p><pre>2023.01.01,17:04,1.069700,1.069740,1.069700,1.069700,0<br>2023.01.01,17:05,1.069730,1.069780,1.069700,1.069710,0<br>2023.01.01,17:06,1.069660,1.069660,1.069660,1.069660,0<br>2023.01.01,17:08,1.069700,1.069740,1.069700,1.069740,0<br>2023.01.01,17:10,1.069750,1.069800,1.069720,1.069720,0<br>...<br>2023.12.29,16:58,1.103660,1.103660,1.103610,1.103610,0</pre><p>The general format of such data is timestamp, followed by open, high, low and closing values for the observed interval. These are then used to construct various variants of candlestick charts.</p><h4>Candles</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9tiifjxLOplk_f01.png" /><figcaption>Single candle</figcaption></figure><p>Let’s get to it, given 1min candles for, make us one day candles:</p><pre>❱ clickhouse local -q &quot;SELECT count() FROM file(DAT_MT_EURUSD_M1_2023.csv)&quot;<br>322638<br><br>❱ clickhouse local -q &quot;DESCRIBE file(DAT_MT_EURUSD_M1_2023.csv)&quot;<br>c1 Nullable(Date)     <br>c2 Nullable(String)     <br>c3 Nullable(Float64)     <br>c4 Nullable(Float64)     <br>c5 Nullable(Float64)     <br>c6 Nullable(Float64)     <br>c7 Nullable(Int64)<br><br>❱ clickhouse local --output-format PrettyCompact -q \<br>&quot;SELECT<br>   toDate(timestamp) AS date,<br>   argMin(open, timestamp) AS open,<br>   max(high) AS max,<br>   min(low) AS min,<br>   argMax(close, timestamp) AS close<br> FROM (<br>   SELECT<br>     toDateTime(toString(c1) || &#39;T&#39; || c2 || &#39;:00&#39;) AS timestamp<br>     c3 AS open,<br>     c4 AS high,<br>     c5 AS low,<br>     c6 as close<br>   FROM file(DAT_MT_EURUSD_M1_2023.csv)<br> )<br> GROUP BY date<br> ORDER BY date<br> LIMIT 5<br>&quot;<br>┌───────date─┬────open─┬─────max─┬─────min─┬──────────────close─┐<br>│ 2023-01-01 │  1.0697 │ 1.07087 │ 1.06788 │            1.06993 │<br>│ 2023-01-02 │ 1.06994 │ 1.06997 │ 1.06499 │ 1.0667200000000001 │<br>│ 2023-01-03 │ 1.06673 │ 1.06706 │ 1.05194 │            1.05727 │<br>│ 2023-01-04 │ 1.05726 │ 1.06354 │ 1.05594 │             1.0604 │<br>│ 2023-01-05 │ 1.06041 │ 1.06288 │ 1.05148 │            1.05243 │<br>└────────────┴─────────┴─────────┴─────────┴────────────────────┘</pre><p>This is almost too cute to be true, our first <em>magic trick</em>, ClickHouse allows us to query CSV file without importing data first. The barrier of entry of using it is very low, and even if clickhouse-local is your only take-out from this article I sincerely hope it could become valuable little tool for daily use.</p><p>Also note the usage of argMin/argMax functions in the above example. ClickHouse has a rich selection of <a href="https://clickhouse.com/docs/en/sql-reference/aggregate-functions/reference">aggregate functions</a>, so you don’t need to write complex aggregations / window function combinations all by yourself.</p><h4>Aggregations</h4><p>And now for the next magical trick. We’ve seen how to summarize minute data with a query, so let’s work on that idea further and make the machine work for us in summarizing candles for other intervals:</p><pre>CREATE TABLE forex (<br>  symbol String,<br>  timestamp DateTime(&#39;UTC&#39;),<br>  open Float64,<br>  high Float64,<br>  low Float64,<br>  close Float64<br>)<br>ENGINE = Null</pre><p>The `Null` table engine is a black-hole where data <em>enters but never leaves</em>. There are now primary keys, ordering or similar, but it defines a sieve which can filter our data.</p><pre>clickhouse local --output-format CSVWithNames -q \<br>&quot;SELECT<br>   &#39;EURUSD&#39; AS symbol,<br>   toDateTime(toString(c1) || &#39;T&#39; || c2 || &#39;:00&#39;) AS timestamp,<br>   c3 AS open,<br>   c4 AS high,<br>   c5 AS low,<br>   c6 as close<br> FROM file(DAT_MT_EURUSD_M1_2023.csv)&quot; &gt; eurusd_2023.csv<br><br>clickhouse client -q &#39;INSERT INTO forex FORMAT CSVWithNames&#39; &lt; eurusd_2023.csv<br><br>clickhouse client -q &#39;SELECT count() FROM forex FORMAT PrettyCompact&#39;<br>┌─count()─┐<br>│       0 │<br>└─────────┘</pre><p>We’ve slightly adapted data to include symbol column, and format the timestamp. However, the data is still not actually saved, this is just a framework for the next step:</p><pre>CREATE TABLE forex_1h_agg<br>(<br>    interval DateTime,<br>    symbol LowCardinality(String),<br>    open AggregateFunction(argMin, Float64, DateTime),<br>    open_timestamp SimpleAggregateFunction(min, DateTime),<br>    close AggregateFunction(argMax, Float64, DateTime),<br>    close_timestamp SimpleAggregateFunction(max, DateTime),<br>    high SimpleAggregateFunction(max, Float64),<br>    low SimpleAggregateFunction(min, Float64)<br>)<br>ENGINE = AggregatingMergeTree()<br>ORDER BY (interval, symbol)<br>PARTITION BY toYear(interval)<br>SETTINGS index_granularity = 8192;<br><br>CREATE MATERIALIZED VIEW forex_1h_agg_mv<br>TO forex_1h_agg<br>AS<br>    SELECT<br>        toStartOfInterval(timestamp, toIntervalHour(1)) AS interval,<br>        symbol,<br>        argMinState(open, timestamp) AS open,<br>        min(timestamp) AS open_timestamp,<br>        argMaxState(close, timestamp) AS close,<br>        max(timestamp) AS close_timestamp,<br>        greatest(max(t.open), max(t.close)) AS high,<br>        least(min(t.open), min(t.close)) AS low<br>    FROM forex t<br>    GROUP BY<br>        symbol,<br>        interval<br>    ORDER BY<br>        symbol ASC,<br>        interval ASC;<br><br>-- Run the above INSERT INTO forex FORMAT CSVWithNames command<br>-- and checkout the results<br><br>SELECT count() FROM forex_1h_agg;<br>┌─count()─┐<br>│    5409 │<br>└─────────┘<br><br>SELECT * FROM forex_1h_agg ORDER BY interval LIMIT 5;<br>┌────────────interval─┬─symbol─┬─open──────┬──────open_timestamp─┬─close────┬─────close_timestamp─┬────high─┬─────low─┐<br>│ 2023-01-01 18:00:00 │ EURUSD │ �H}�c      │ 2023-01-01 18:04:00 │  ��c │ 2023-01-01 18:59:00 │ 1.07066 │ 1.06788 │<br>│ 2023-01-01 19:00:00 │ EURUSD │ �<br>                                  �� ʱc     │ 2023-01-01 19:00:00 │ W[���c    │ 2023-01-01 19:59:00 │ 1.07047 │ 1.06829 │<br>│ 2023-01-01 20:00:00 │ EURUSD │ ;��0رc    │ 2023-01-01 20:00:00 │ �[ ��   │ 2023-01-01 20:59:00 │ 1.07041 │ 1.06924 │<br>│ 2023-01-01 21:00:00 │ EURUSD │ Ut&lt;f �@� │ 2023-01-01 21:00:00 │ �����c      │ 2023-01-01 21:59:00 │ 1.07047 │ 1.06962 │<br>│ 2023-01-01 22:00:00 │ EURUSD │ ����P�c      │ 2023-01-01 22:00:00 │ {���$c     │ 2023-01-01 22:59:00 │ 1.06977 │ 1.06871 │<br>└─────────────────────┴────────┴───────────┴─────────────────────┴──────────┴─────────────────────┴─────────┴─────────┘<br><br>-- What&#39;s up with that output, binary? Let&#39;s fix it:<br><br>SELECT<br>    interval,<br>    symbol,<br>    argMinMerge(open) AS open,<br>    max(high) AS high,<br>    min(low) AS low,<br>    argMaxMerge(close) AS close<br>FROM forex_1h_agg<br>GROUP BY<br>    interval,<br>    symbol<br>ORDER BY interval ASC<br>LIMIT 5<br><br>Query id: 5f5be484-e89f-426b-ac4e-6fb4d36cb9c3<br><br>┌────────────interval─┬─symbol─┬────open─┬────high─┬─────low─┬───close─┐<br>│ 2023-01-01 18:00:00 │ EURUSD │  1.0697 │ 1.07066 │ 1.06788 │ 1.06929 │<br>│ 2023-01-01 19:00:00 │ EURUSD │ 1.06896 │ 1.07047 │ 1.06829 │ 1.07005 │<br>│ 2023-01-01 20:00:00 │ EURUSD │ 1.07007 │ 1.07041 │ 1.06924 │  1.0704 │<br>│ 2023-01-01 21:00:00 │ EURUSD │ 1.07041 │ 1.07047 │ 1.06962 │ 1.06977 │<br>│ 2023-01-01 22:00:00 │ EURUSD │ 1.06977 │ 1.06977 │ 1.06871 │ 1.06904 │<br>└─────────────────────┴────────┴─────────┴─────────┴─────────┴─────────</pre><p>This a lot to unpack, have some coffee now.</p><p>First, we’ve created a table to hold our aggregated data — forex_1h_agg, and then used a few interesting ClickHouse features to sculpt the data:</p><ul><li>We’ve used <a href="https://clickhouse.com/docs/en/sql-reference/data-types/lowcardinality">LowCardinality</a> to encode the information about symbol column. Usually, it’s a three letter (or more) character column. ClickHouse doesn’t really have the usual database indices (b-tree, hash, etc), but <a href="https://clickhouse.com/docs/en/optimize/sparse-primary-indexes">sparse primary</a> and <a href="https://clickhouse.com/docs/en/optimize/skipping-indexes">data skipping indices</a>. Since we know that symbol set is limited in number (at most a few thousands symbols), we can instruct ClickHouse to encode the data in more efficient dictionary format (ClickHouse even has enums, but for string data, LowCardinality is better).</li><li>The next revelation is about <a href="https://clickhouse.com/docs/en/sql-reference/data-types/aggregatefunction">Aggregate Function</a> column type. This is the real deal which distinguishes ClickHouse from other databases.<br>open AggregateFunction(argMin, Float64, DateTime)<br>Here we’ve instructed ClickHouse that our open column is of the Float64 type, but the data is not final (i.e. the column will be updated). In addition to that, we’ve noted that the data uses argMin(Float64, DateTime) aggregation (i.e. the open value is dependent on other column of the DateTimetype)<br>With this, ClickHouse is able to deduce when to update the column depending on the value of (open, timestamp) pair (for given time interval, we always want for “oldest” value to be open value, whenever the data is inserted into ClickHouse.</li><li>For easier debugging, we’ve saved the timestamp of open value ourselves in open_timestamp column. The aggregation used here is “simple”, so we have a special case for Aggregate Function, so ClickHouse can persist the data in more convenient format (the storage type of max(Float64)column is equivalent to plainFloat64.</li><li>The resulting data set for hourly candles has shrank to 5 thousand rows. As we insert the data, the ClickHouse filters and aggregates the data <em>on the fly</em>. The insert batch is also aggregated in the Materialized View query, and then it is aggregated again when merging with the data on the disk, hence the AggregatingMergeTree table type. If the insert batch contained 100k rows with nanosecond timestamp, our MV query would aggregate it away into per minute chunks before hitting the forex_1h_agg backing table (and disk).</li><li>Our forex_1h_agg_mv <a href="https://clickhouse.com/docs/en/guides/developer/cascading-materialized-views">materialized view</a> is triggered on every insert into forex_1h table. That allows us to use Nulltable engine, and avoid saving unecessary data (we only care about OHLC)</li><li>We’ve also observed that the AggregatedFunction data is in some kind of binary format (it looks like it is space efficient, since it is in our case String + DateTime), and that for exposing the data we need to add Mergesuffix to unveil the value (argMinMerge/argMaxMerge).</li></ul><p>To recap, you can ingest millions of data points into ClickHouse, reduce the data set on the fly, and persist the aggregations. The fine folks at <a href="https://www.timescale.com/">Timescale </a>DB call this <a href="https://docs.timescale.com/use-timescale/latest/continuous-aggregates/">continuous aggregatates</a> which beautifully describes the concept.</p><p>If it is not already obvious, this means that ClickHouse can also be used as a <a href="https://en.wikipedia.org/wiki/Time_series_database">time series database</a>. Where you can save all your analytical data in one place and query it efficiently.</p><h4>Charts</h4><p>The last, but not the least, before we fall to sleep, lets fire up our candles!<br>If you have used a time series database before, you’ve probably used <a href="https://grafana.com/">Grafana</a>, a wonderful visualization and alerting tool. It has good support for ClickHouse data source and it also has candlestick charts out of the box.</p><p>Our data table is slightly complicated for Grafana QueryBuilder, but we can dive in with a custom query.</p><pre>SELECT<br>    $__timeInterval(interval) as timestamp,<br>    argMinMerge(open) AS open,<br>    toFloat64(max(high)) AS high,<br>    toFloat64(min(low)) AS low,<br>    argMaxMerge(close) AS close<br>FROM &quot;default&quot;.&quot;forex_1h_agg&quot;<br>WHERE ( timestamp &gt;= $__fromTime AND timestamp &lt;= $__toTime )<br>GROUP BY<br>    timestamp,<br>    symbol<br>ORDER BY timestamp ASC<br>LIMIT 1000</pre><p>The dollar variables are for templating, so you can select interval in the Grafana dashboard; For some reason, Grafana could not detect our max(high)and min(low)so we had to typecast it manually.</p><p>The end result is surprisingly good</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1nsghiJFchFFXn_I03jA8g.png" /><figcaption>2023-01-02T18:00:00 — 2023-01-03T18:00:00</figcaption></figure><p>Grafana can also alert on various conditions etc, so it is just perfect for exploring and observing the live data.</p><h3>What’s next?</h3><p>We’ve just begin our journey with ClickHouse:</p><ol><li>Learn about ClickHouse!</li><li>Charts! ← <em>you are here</em></li><li>…</li><li>…</li><li>Profit(s)!</li></ol><p>Before reaching our goal, we’ll have two more side quests so you learn how to stitch a useful real life application:</p><ul><li>How to feed the ClickHouse machine with Kafka</li><li>How to cross the Kafka message stream with Java</li></ul><p>Both technologies, Kafka and Java are enterprise(tm), big data, and would compliment everyone&#39;s resume.</p><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged remote-optimized venture builder specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>If you want to work with us on your startup feel free to contact us at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad6dc56abf67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/using-clickhouse-for-charts-profits-ad6dc56abf67">Using ClickHouse for Charts &amp; Profits</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI’s impact: Transforming IT Developers to Engineers]]></title>
            <link>https://medium.com/mop-developers/ais-impact-transforming-it-developers-to-engineers-ebba37a972d1?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/ebba37a972d1</guid>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Armin Cicak]]></dc:creator>
            <pubDate>Wed, 27 Mar 2024 10:27:04 GMT</pubDate>
            <atom:updated>2024-03-27T10:27:03.860Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IsN6j3WT9CWOQtyrS7D8JQ.jpeg" /><figcaption>What does the future hold? Continue reading.</figcaption></figure><p><strong>This world needs more engineers!</strong></p><p>It sounds like a concern raised during the Industrial Revolution in the nineteenth century, yet it holds the same relevance in the era of the digital revolution facing the world today.</p><p>As a project manager with a diverse background spanning engineering, design, marketing, and IT project management on global scales within international teams and clients, I draw upon various experiences to articulate compelling arguments on this one. My career journey, which also included music, acting, and teaching mathematics, provided valuable insights into the significance of multidisciplinary approaches to solving challenges and issues. Just as my most successful students excelled not only in mathematics but also in various pursuits such as music, sports, or art, the same principle applies to the realm of IT projects.</p><p>The onset of the COVID-19 pandemic catalyzed an unprecedented surge in demand for digitalization and IT solutions. In that environment, IT companies have been urged for rapid response in hires and an enormous coding speed that resulted in the lack of quality and perfection in teams. However, as the post-pandemic landscape emerges, there arises a compelling need to transform the basic IT approach and cultivate enriched value for clients and IT roles alike.</p><p>Elevating the standards of daily development work is essential (but not exclusive) for teams and projects alike to gain new value for the clients and users. Transitioning from the conventional roles of junior, medior, or senior developer to the more encompassing engineering mindset is pivotal. This paradigm shift underscores the importance of infusing additional value into the people, business, products, projects, apps and features, thereby ensuring competitiveness and relevance in an evolving market with different requests than we experienced before.</p><figure><img alt="The image is a vibrant abstract depiction of change in the tech industry, featuring a central spiral and assorted geometric shapes that convey motion and progress." src="https://cdn-images-1.medium.com/max/1024/1*5QeMV2uENZJueFS0xCQhNw.jpeg" /><figcaption>The only constant in this ever-dynamic work environment is change.</figcaption></figure><p>While this discourse may extend across various IT areas such as project management, product design, product development etc., this article focuses merely on the development segment, encompassing code production, testing, and deployment processes.</p><p>In this transformative journey, the synthesis of diverse perspectives listed below emerges as a cornerstone for success in changing the paradigm of doing pure development and being a developer/coder, to engineering and being a modern engineer.</p><p><strong><em>Engineering dynamics and approach</em></strong></p><p>Engineers are engaged in the act of construction rather than pure coding or development. When contemplating the creation of an ideal family home, considerations encompass not only the physical structure but also its placement within the locality, the surrounding community, the neighborhood’s ambiance, the standard of living it affords, potential hazards, the expenses incurred, and the suitability for raising a family. Ultimately, the focus centers on the overall excellence, safety, and functionality of the dwelling.</p><p>Similarly, a comparable contrast emerges between the methodologies of engineering and pure development. IT engineers undertake<strong> a holistic approach</strong>, contemplating the architectural framework, the seamless integration of features or functionalities within the broader application ecosystem, the projected costs, user satisfaction, potential challenges, and risk assessment associated with implementing the solution. Conversely, developers concentrate primarily on the coding aspect, emphasizing the quality of the code and the fulfillment of acceptance criteria delineated in the user story.</p><p><strong><em>Engineering engagement and development</em></strong></p><p>Engineers are not merely executing and implementing tasks but are actively engaged in questioning and exploring various implementation methodologies and solutions. These inquiries often lead to divergent perspectives and innovative, <strong>out-of-the-box thinking</strong>, occasionally challenging the proposed consensus among product designers, project managers, product owners, and engineering leads during the development of features or functionalities.</p><p>The iterative process of questioning not only facilitates critical analysis but also fosters a culture of collaboration and constructive discourse within the team. It encourages the exploration of alternative solutions and promotes a deeper understanding of the underlying challenges and opportunities inherent in the project. By embracing this approach, engineers contribute to the evolution of the product development lifecycle, enriching it with diverse perspectives and novel insights that ultimately enhance the quality and efficacy of the final deliverable.</p><p>In contrast to engineers who question and explore methodologies and proposed solutions, developers often follow directives without probing deeper. While they <strong>request for the established clear acceptance criteria</strong> before the development starts to guide their actions and decisions, they often fall short in documenting the implementation approach and demonstrating the implemented feature to their peers. This divergence can lead to unanticipated challenges and project failures.</p><blockquote>Simply executing orders without critical inquiry can compromise project integrity and success.</blockquote><p>Therefore, fostering a collaborative and inquisitive engineering environment is indeed crucial for project success.</p><p><strong><em>Engineering Versatility and Skillset</em></strong></p><p>Engineers summarise the concept of <strong>T-shaped individuals</strong>, embodying a breadth of knowledge and skills spanning diverse domains while also possessing profound expertise in a particular area. Their versatility and proactivity allow them to navigate various FE and BE disciplines effectively, even in domains such as DevOps, product development, or security, where they may encounter unfamiliar terrain. Embracing a culture of open-mindedness, engineers readily absorb new insights and perspectives, recognizing the value of continuous learning for both project advancement and personal growth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pUBxerTKqNzcpACJwMGOww.jpeg" /></figure><p>In contrast, <strong>I-shaped developers</strong> tend to specialize deeply in specific domains while potentially lacking exposure to other areas. Their focus is typically directed towards honing expertise within their chosen field, often leading to a narrower scope of (or no any) knowledge and lagging to entertain outside their basic domain.</p><p><strong><em>Process adoption and data valorization</em></strong></p><p>Engineers increasingly recognize the advantages of implementing structured processes and robust project management methodologies.</p><blockquote>They grasp the significance of diligent planning, understanding the pivotal role that project data, encompassing estimations, work logs, and efforts, plays in informing future actions and optimizing outcomes.</blockquote><p>Conversely, developers may exhibit reluctance towards embracing changes in processes, expressing reservations about the visibility of their work within the project and the additional responsibilities associated with reporting to project managers or engaging in tasks beyond the realm of pure development or coding. They may harbor apprehensions about deviating from familiar, entrenched processes, even when confronted with the shortcomings of existing methods or the potential benefits of newly introduced frameworks.</p><p><strong><em>Experimentation</em></strong></p><p>Engineers prioritize experimentation over-reliance on legacy experiences, recognizing that such conventional approaches may yield suboptimal outcomes. By embracing experimentation, they explore a spectrum of solutions, acknowledging that some may succeed while others may fail, yet each endeavor contributes valuable insights to the challenges they confront.</p><blockquote>Engineers encourage a balance between innovation and tradition within their teams.</blockquote><p>By fostering a culture that encourages calculated risk-taking and embraces the potential for transformative change, they harness the collective ingenuity to drive continuous improvement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P5hcPmsP754pv-snkjdumA.jpeg" /><figcaption><em>Engineers favor experimentation, challenging legacy methods for optimal outcomes.</em></figcaption></figure><p>Conversely, developers are often drawn to familiar and established processes, preferring the comfort of the “safe zone” despite the potential for recurring mistakes, coding challenges, and stagnation in quality over time.</p><p><strong><em>Embracing Failure</em></strong></p><p>In the realm of achievement and fear of failure lies the contrast between engineers’ fearless pursuit of innovation and developers’ aversion to failure. While experimentation may occasionally lead to setbacks, the final output tends to be more refined and impactful. Engineers embrace failure as an aspect of the learning journey, leveraging each setback as an opportunity for growth, team-wide acknowledgment, and collective problem-solving.</p><figure><img alt="An illustration portraying a lone traveler at the base of a vast, mountainous terrain, evoking the hardships of a challenging journey." src="https://cdn-images-1.medium.com/max/1024/1*riULjnHMao-ikpvZlqXPvg.jpeg" /><figcaption>Failure is a just small part of your journey. No stopping now.</figcaption></figure><p>Conversely, developers often recoil from failure, endeavoring to hide it rather than confronting and addressing it openly within the team. This reluctance to acknowledge failure can breed uncertainty and exacerbate challenges, hindering the team’s ability to adapt and evolve effectively.</p><p>This culture that values transparency, resilience, and continuous learning, can help teams transform setbacks into stepping stones towards innovation and sustained success.</p><h3><strong>Conclusion</strong></h3><p><strong>This world needs more engineers!</strong> In today’s dynamic landscape, the demand for IT engineers is enormous. Engineers embody a mindset of inquiry rather than mere execution, embracing experimentation over adherence to conventional methods, and fostering innovation that accepts potential failure. However, this narrative overlooks other complexities inherent in engineering projects, including costs, expertise, project maturity, budget, and stakeholder involvement.</p><p>It is imperative for all teams, whether working in startup environments or established projects, to recognize the need for a shift in approach towards their work and attitude to create value-driven solutions for stakeholders and clients. While hiring growth-minded engineers may pose challenges compared to hiring developers, the ultimate benefit to the project outweighs the initial hurdles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ahJsD8isjYjXQua9fy33A.png" /></figure><p>From a project management standpoint, the choice between a team of developers or engineers depends on the nature of the project and the leadership style employed. Nurturing an environment that encourages experimentation and fosters excitement among team members is akin to raising an adventurous child, which may pose challenges but ultimately reflects the difference between being a boss and being a leader in guiding both the project and the team toward success.</p><p><strong><em>…</em></strong></p><p><em>Ministry of Programming is a supercharged startup studio specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times. Twice.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>Building your next startup? We would love to hear more. If you want to work with us on your startup feel free to reach out at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebba37a972d1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/ais-impact-transforming-it-developers-to-engineers-ebba37a972d1">AI’s impact: Transforming IT Developers to Engineers</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is VPN and why should I use it?]]></title>
            <link>https://medium.com/mop-developers/what-is-vpn-and-why-should-i-use-it-c9716a3edca5?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/c9716a3edca5</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[vpn-service]]></category>
            <category><![CDATA[vpn]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Kenan Jazic]]></dc:creator>
            <pubDate>Mon, 06 Nov 2023 12:54:54 GMT</pubDate>
            <atom:updated>2024-03-13T08:29:27.244Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fmEtPAvnSc0pXTjtpRLTmQ.jpeg" /><figcaption>Why’s VPN something important to think about?</figcaption></figure><p>If you’ve ever wondered about this question, then this article contains all the information you need to know about VPNs.</p><p>But let’s start with the history. VPN or Virtual Private Network is technology that was first used in 1996 by a Microsoft’s employee who developed PPTP (Point to Point Tunneling Protocol). Since 1996, VPN technology has seen widespread use and continuous development in various forms.</p><p>Reason for developing VPN technology is to reduce costs from telecom operators for dedicated lines which cost a fortune in the 90’s. If you had a company with a lot of branch offices, something like supermarket chains or banks or government offices to connect them all together that only for dedicated lines could cost a lot of money. Business was looking for a solution to reduce that costs and VPN was born.</p><p>VPN technology uses public internet connection to connect two or multiple offices ensuring end-to-end encryption for data transferred over the public network.</p><p>The best practice for companies with multiple branches and centralized data centers at their headquarters is to implement IPSec site-to-site VPN solutions. Multiple VPN technologies and methods can be deployed in these scenarios, including::</p><ul><li>DMVPN</li><li>DVTI VPN</li><li>Flex VPN</li><li>GetVPN</li><li>Client IPSec VPN</li></ul><p>These types of VPN use two phases for encryption traffic.</p><p><em>Phase 1</em> — Negotiate exchange of proposals for how to authenticate and secure the channel.</p><p><em>Phase 2</em> — Negotiate security associations (SAs) to secure the data that traverses through the IPsec tunnel.</p><p>When deploying this kind of solution you should use the most secure protocols supported by your devices.</p><p>The best practice for IT professionals is to employ client-based VPN solutions enabling them to access data centers regardless of their location, whether the data centers are in the cloud or on-premises. This kind of VPN solution is the most suitable for IT professionals as it allows them the access from anywhere and at any time, facilitating their tasks efficiently.</p><p>Client based VPN solutions expanded during the COVID19 pandemic so other employees who had not used VPN previously needed to configure it and use them for remote work.</p><p>Client based VPN is a virtual private network created between a single user and a remote network. There’s often an application involved to make the connection. In most scenarios, the user manually starts the VPN client and authenticates with a username and password. Client based VPN solutions mostly use SSL protocol to encrypt traffic. But it is not secured enough as we know username and password is not secure enough for authentication of users. So for this kind of connection to be secured it should be mandatory to use multi factor authentication for all users.</p><p>In order to improve security, organisations and companies should ensure that all resources accessed by employees, including admin tools, databases, and servers, are exclusively accessed through a VPN. Only resources that are customer-related should be accessible publicly using HTTPS or some other secure protocol while access to all unnecessary ports should be restricted.</p><p>If we use the line of least resistance and ignore VPN and access publicly to our internal resources from anywhere, we expose our resources to risk to be hacked and misused. If we do not get rid of technical debt and do not upgrade on a regular basis we are at very high security risk. If we can access internal resources from anywhere, means that anyone, including unauthorized individuals, can potentially access them. If we think that credentials will stop hackers from accessing publicly accessible resources. We are wrong and probably hacked already.</p><p>So, the answer to the question why should I use VPN is to securely access remote resources from anywhere.</p><p>Protect your resources and ensure security by using a VPN.. Mitigate security risk and limit or restrict public access to company resources.</p><h3><strong>Is VPN everything we need for security </strong>or are there more secure alternatives available?</h3><p>Is VPN everything we need for secure access to remote resources? No, it is not.</p><p>It is just a good start and secure way to access remote resources. <br>However, there are a few security questions to consider:</p><ul><li>Should all users access all resources in the company? No</li><li>Is there a possible malicious user in the company? Yes</li><li>Do all remote workers have protected workstations? No</li></ul><p>From a security perspective, we should not trust anyone and that is how the concept of Zero Trust Network Access is born. Zero Trust Network Access (ZTNA) is a security framework that emphasizes the principle of “never trust, always verify” in network communications. It’s designed to enhance cybersecurity by providing granular control over access to applications and resources, regardless of whether users are inside or outside the corporate network.</p><p>With Zero Trust Network Access or ZTNA, IT teams like DevSecOps or Security teams have power to control this kind of access to resources.</p><p>ZTNA solutions give you the possibility to granulate access to resources by user or user group. So, for example, a legal team has access only to legal applications or trading team has access only to trading applications by needed ports and nothing more. With this granularity we can effectively prevent malicious users from accessing resources they shouldn’t have access to.</p><p>ZTNA solutions also give you the ability to restrict access to unpatched or not updated devices so it can control which device is permitted to access. With this capability, we can prevent vulnerable devices from accessing critical resources, thereby reducing security risks.</p><p>From a startup perspective, VPN usage should be mandatory for all engineers and developers who need to access infrastructure.</p><p>ZTNA solutions should be considered as security enhancing which can benefit both startups and employees. Some benefits for startups can be:</p><ul><li><strong>Scalability:</strong> Startups often experience rapid growth. ZTNA solutions are designed to scale as your organization expands, ensuring that security remains effective and manageable.</li><li><strong>Compliance and Regulations:</strong> ZTNA can help startups meet compliance requirements by providing robust security controls and audit trails for access to sensitive data and applications. It is a great solution for fintech startups.</li><li><strong>Visibility and Monitoring:</strong> ZTNA solutions often provide detailed visibility into user behavior and network traffic. This visibility aids in identifying unusual activities and potential threats.</li><li><strong>Cost Efficiency:</strong> ZTNA reduces the need for hardware and infrastructure investments associated with traditional perimeter security solutions. This can be especially beneficial for startups with limited budgets.</li><li><strong>User Experience</strong>: ZTNA solutions often provide a seamless and user-friendly experience. Employees and partners can access resources without the hassle of connecting to a VPN or dealing with multiple authentication steps.</li><li><strong>Prevent Data Loss/Leakage: </strong>ZTNA solutions provide granular access, allowing us to prevent code and data leakage from full startup solutions or ideas. Engineers or contractors should only have access to the services they are actively working on.</li></ul><p>It’s important to note that while ZTNA offers significant benefits, it’s not a one-size-fits-all solution. Startups should carefully evaluate their security needs, the types of applications and data they handle, and the level of control required before implementing ZTNA.</p><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged startup studio specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times. Twice.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>Building your next startup? We would love to hear more. If you want to work with us on your startup feel free to reach out at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9716a3edca5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/what-is-vpn-and-why-should-i-use-it-c9716a3edca5">What is VPN and why should I use it?</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to nurture psychological safety as a Product Manager]]></title>
            <link>https://medium.com/mop-developers/how-to-nurture-psychological-safety-as-a-product-manager-c5b7440b4b95?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/c5b7440b4b95</guid>
            <category><![CDATA[product-management]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[company-culture]]></category>
            <category><![CDATA[psychological-safety]]></category>
            <dc:creator><![CDATA[Ejub Hadžić]]></dc:creator>
            <pubDate>Mon, 16 Oct 2023 08:27:17 GMT</pubDate>
            <atom:updated>2024-03-13T08:31:01.128Z</atom:updated>
            <content:encoded><![CDATA[<h3>In the midst of tech layoffs, I’ve come to believe that psychological safety isn’t just the lifeboat — it’s the compass that navigates us through the storm.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yNsyqaUvkUM-t1rNo32b_Q.jpeg" /></figure><p>But what does navigating through a storm in the tech industry mean?</p><p>How does one find their compass amidst the chaos?</p><p>Can psychological safety truly guide us through the turbulence, or is it just another buzzword? And if it is indeed our compass, how do we read it, and more importantly, how do we follow it?</p><p>Let’s delve into these questions as we explore the concept of psychological safety from a product manager’s perspective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uo_anoqbAzofPu3_KBcnag.png" /><figcaption>Midjourney imagining a lifeboat navigating the storm towards the calmer waters</figcaption></figure><p>From a personal perspective, as a product manager with over six years of experience in both the startup and agency world, I’ve seen my fair share of turbulence. I’ve been a part of teams that had the drive to push forward no matter what, and I’ve been a part of organizations that made you feel like a cog in a machine — easily replaceable. And now that layoffs are becoming a common headline, the tech industry seems to be in a constant state of flux.</p><p><strong>These numbers speak for themselves:</strong></p><ul><li>January: 84,714 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#janlayoffs">see all January 2023 Tech Layoffs</a></li><li>February: 36,491 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#feblayoffs">see all February 2023 Tech Layoffs</a></li><li>March: 37,109 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#marchlayoffs">see all March 2023 Tech Layoffs</a></li><li>April: 17,926 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#aprillayoffs">see all April 2023 Tech Layoffs</a></li><li>May: 14,555 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#maylayoffs">see all May 2023 Tech Layoffs</a></li><li>June: 10,524 employees laid off — <a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L#junelayoffs">see all June 2023 Tech Layoffs</a></li></ul><p><em>Source: </em><a href="https://techcrunch.com/2023/07/20/tech-industry-layoffs-2023/?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAACwsTJ7LOks6TfadCPiz7eLkAadQygV9WbPeNt_pvlveGjetaxk6madMYPvgEwtF_L7wD1BI47ur8FonlrRoj9LKFNy1GSE3no7oJ7gNNqUNmlN7ZszxqO3kOtPSG4dZ9Nl6D3Q5_D-XnNZOjrlxGHTOr8hT14fOW0-i5cfD_r2L"><em>TechCrunch</em></a></p><figure><img alt="2023 Tech layoffs by TechCrunch" src="https://cdn-images-1.medium.com/max/1024/1*YuAHh09zCHf-M1HPLoNh1w.png" /><figcaption><em>Source: </em><a href="https://techcrunch.com/2023/05/18/tech-industry-layoffs/"><em>TechCrunch</em></a><em>, </em><a href="https://layoffs.fyi/"><em>Layoffs.fyi</em></a><em> • This chart was last updated July 6, 2023. Some numbers in this chart may have been estimated based off percent of employees being laid off and company headcount that’s publicly available. </em>Data visualization by <a href="https://twitter.com/mirandahalpern">Miranda Halpern</a>, created with <a href="https://flourish.studio/">Flourish</a></figcaption></figure><p>And if you’re by any chance on (business) social networks such as LinkedIn, more probably than not, you’ve seen some posts highlighting layoff situations.</p><p>Yet, amidst this chaos, I’ve started to see a beacon of stability: psychological safety. More than a buzzword, it’s often mentioned as a significant factor in creating high-performing teams, according to <a href="https://rework.withgoogle.com/print/guides/5721312655835136/#:~:text=In%20a%20team%20with%20high,or%20offering%20a%20new%20idea.">Google’s research</a>. And I’m starting to see why.</p><h3>Endurance vs. Suffering</h3><p>I used to have a phrase written beside my bed: “He conquers who endures.”</p><p>And I still stand by it. Another version uses “suffers” instead of “endures,” which conveys a completely different point for me. Enduring something means there will be storms and tough situations, but it does not serve as an excuse to settle, suffer, and do nothing. In order to help, lead, and manage others you need to the same for yourself first. A fellow PM, and a colleague of mine neatly described the state of mind you should strive for:</p><blockquote>No matter what happens, if you give your best effort, energy, and will, everything that unfolds is simply paving the way for even better things ahead. With this mindset, nothing can bring you down.</blockquote><p>Referring back to the first sentence, it’s easier to endure a storm if you have a compass that navigates you through it.</p><p>So, how do we navigate this storm of uncertainty?</p><p>How do we endure and not merely suffer?</p><p>The answer, I believe, lies in the very heart of our teams and organizations. It’s about creating an environment where people feel safe to take risks, voice their ideas, and be their authentic selves. It’s about fostering psychological safety. Now, you might be wondering, “That sounds great, but how do we actually do that, especially in the face of layoffs and economic uncertainty?”</p><figure><img alt="Midjourney imagining Psychological Safety" src="https://cdn-images-1.medium.com/max/1024/1*AWB9_7DVlAGFrCjEw0QDpw.png" /><figcaption>Midjourney Imagining Psychological Safety</figcaption></figure><p>As I understand it, psychological safety is the bedrock of a healthy work environment. And creating and maintaining it during tech layoffs can feel like a Herculean task. The looming threat of layoffs can create a tense atmosphere where employees focus more on survival than innovation. But here’s the paradox: it’s precisely in these challenging times that psychological safety becomes even more critical.</p><p>It requires open empathy, a team dynamic that values every voice, clear communication, and managing expectations of all stakeholders. But most importantly, it’s about acknowledging the human element in every employee, especially when uncertainty is ever-increasing, and every move may be perceived as risky.</p><blockquote>When you think about all these moving parts, it’s safe to argue that a product manager&#39;s role in fostering psychological safety is multifaceted — or in other words: <strong>to Be.</strong></blockquote><p>To be what? Let’s briefly expand on each point:</p><h3>Be a Bridge</h3><p>Firstly, you’ll serve as a bridge between various stakeholders — from developers and designers to marketing and sales teams. It’s your responsibility to ensure that everyone feels heard and that their ideas and concerns are taken into account. This requires open communication, active listening, and empathy, which can help build trust and foster a sense of safety within the team.</p><p>I usually do these by having weekly or biweekly 30-minute 1:1 sessions with each team member. This is their opportunity to set an agenda and freely discuss their issues, challenges, wants, and needs. My primary focus here is their mental health and nurturing an environment of togetherness.</p><h3>Be a Beacon</h3><p>Secondly, aim to provide clear direction and expectations. This involves communicating the broader vision and goals of the project and how each team member’s work contributes to these.</p><blockquote><em>When team members understand the ‘why’ behind their tasks, it can enhance their sense of purpose and motivation.</em></blockquote><p>I have often fallen prey to the idea that a clear roadmap, organized backlog, and nicely defined epics, stories, tasks, etc., are enough for everyone to understand the context. While these are certainly important factors, they represent only the last part of the “alignment” pyramid.</p><p><a href="https://www.lennysnewsletter.com/p/mission-vision-strategy-goals-roadmap">This article</a> from Lenny’s Newsletter provides a great example of what is actually needed for everyone to understand the whole context.</p><figure><img alt="Product Contect Pyramid. Source: Lenny’s Newsletter" src="https://cdn-images-1.medium.com/max/1024/1*jeRA48msHcD2nBY3AMcilw.png" /><figcaption>Source: <a href="https://www.lennysnewsletter.com/p/mission-vision-strategy-goals-roadmap">Lenny’s Newsletter</a></figcaption></figure><p>Product documentation that starts from a Mission and Vision that is clearly cascaded into the roadmap and stories is your superpower. However, it’s important to note that documentation alone won’t have an impact — you need to communicate and remind the team about the whole pyramid regularly.</p><h3>Be a Catalyst</h3><p>Thirdly, strive to create an environment where failure is not feared but seen as a learning opportunity. This means encouraging risk-taking and innovation within loosely defined boundaries (to avoid points of no returns), and when mistakes happen (as they inevitably do), focusing on the lessons learned rather than placing blame.</p><p>The important premise here is that you should prioritize being a beacon.</p><p>Why?</p><p>Experimenting, innovating, and going the extra mile are generally good things, but you must ensure that you are moving in the right direction. A team not aligned on the direction or vision can often become scattered and confused, with everyone going their own way. However, when the team ventures into uncharted areas, the key is to focus on learning from the experience through retrospectives, experiment reviews, or any other form of post-mortem. Regardless of the method, while talking is good, documenting and putting notes and lessons learned on paper gives clarity and accountability.</p><h3>Be a Celebrator</h3><p>Lastly, believe in the importance of acknowledging and celebrating successes, no matter how small. Recognizing the hard work and achievements of the team not only boosts morale but also reinforces a positive and supportive team culture.</p><p>Over the years, I have learned that small, genuine gestures of public recognition can go a long way. We all live in our own versions of the world that becomes that much better when others openly acknowledge the effort and work we have done with the best intentions in our minds. Communicating progress, even small steps towards the bigger goal creates a positively encouraging environment that is more appealing to everyone involved.</p><figure><img alt="Midjourney imagining a network of multiple nodes (roles) connected together representing the wide range of skills a PM should have." src="https://cdn-images-1.medium.com/max/1024/1*QdVwWLb189k_rT-nH9l8Fw.png" /><figcaption>Midjourney imagines a network of multiple nodes (roles) connected, representing the wide range of skills a PM should have.</figcaption></figure><p>In essence, it’s a challenging but rewarding role, and it’s crucial in building high-performing teams, especially in these uncertain times.</p><h3>What About Startup Founders?</h3><h4><em>Navigating Psychological Safety at the Helm</em></h4><p>Psychological safety is a two-way street. Those experiencing it and those nurturing it.</p><p>And that’s why it’s important for startup founders to take this topic seriously and lead the way.</p><p>As a startup founder, your role in fostering psychological safety has an even greater significance. You’re not just a bridge, a catalyst, a beacon, or a celebrator — you’re the architect of the company culture.</p><p>Creating a psychologically safe environment starts from the top. It’s about setting the tone and shaping the narrative. By being transparent about the company&#39;s challenges and vision, you can help your team see beyond the immediate turbulence. Because in a startup, every challenge is an opportunity for growth.</p><h3>Let’s wrap it up.</h3><p>In the face of tech layoffs, psychological safety is more than just a safety net — it could be a launchpad — it all depends on how you choose to look at it. It’s what could allow teams to weather the storm and emerge stronger on the other side. As we navigate these uncertain times, let’s remember to prioritize psychological safety.</p><p>Because it’s not just about weathering the storm — it’s about learning to thrive amidst the chaos. And let’s not forget; it’s not a question of whether we’ll face another storm but how we’ll react when we do. It’s reassuring to be surrounded by people who feel safe when the sea is calm and when the storm hits.</p><p>After all, if we’ve weathered it once, who’s to say we can’t do it again?</p><p>Keep sailing, my friends.</p><h4><strong>Key Takeaways:</strong></h4><ol><li>According to <a href="https://rework.withgoogle.com/print/guides/5721312655835136/#:~:text=In%20a%20team%20with%20high,or%20offering%20a%20new%20idea.">Google&#39;s research</a>, psychological safety is a significant factor in creating high-performing teams.</li><li>The role of a product manager is crucial in fostering psychological safety during tough times.</li><li>Clear communication, empathy, and valuing every voice are essential for maintaining psychological safety.</li><li>Celebrating successes, no matter how small, can boost morale and reinforce a positive team culture.</li><li>Psychological safety can transform a challenging situation into a launchpad for growth and resilience.</li><li>Lastly, great things can be achieved if we root our collaboration and relationship as humans first, then business colleagues.</li></ol><h3>… just one more thing</h3><p>In this nurturing environment where individuals are prioritized, heard, and genuinely cared for, you might find yourself questioning, “Where do you draw the line? What are the boundaries?”</p><p>Balancing these elements is indeed a delicate act. How do you ensure that in the pursuit of psychological safety, you don’t tip the scales too far and inadvertently create a counterproductive environment?</p><p>It’s a thought-provoking question, and one that invites us all to continually reflect, adapt, and strive for that ideal balance in our workplaces. And the one I will cover in more detail in my next article soon.</p><p>Until then — keep sailing.</p><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged startup studio specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times. Twice.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>Building your next startup? We would love to hear more. If you want to work with us on your startup feel free to reach out at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5b7440b4b95" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/how-to-nurture-psychological-safety-as-a-product-manager-c5b7440b4b95">How to nurture psychological safety as a Product Manager</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How man-in-the-middle saved the day]]></title>
            <link>https://medium.com/mop-developers/how-man-in-the-middle-saved-the-day-f1eef6506d2e?source=rss----5d6f05ead37f---4</link>
            <guid isPermaLink="false">https://medium.com/p/f1eef6506d2e</guid>
            <category><![CDATA[documentation]]></category>
            <category><![CDATA[task-management]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[communication]]></category>
            <category><![CDATA[android-app-development]]></category>
            <dc:creator><![CDATA[Adnan Sinanovic]]></dc:creator>
            <pubDate>Tue, 13 Jun 2023 09:46:15 GMT</pubDate>
            <atom:updated>2024-03-13T08:29:55.939Z</atom:updated>
            <content:encoded><![CDATA[<h4>True software development horror experience</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9TCotKnPDaGiP1kjlx2H-g.jpeg" /></figure><h3>Innocent beginning</h3><p>Many, many years ago I worked with Windows desktop C♯ and C++ apps. The app was accessible through public interfaces, which we internally called Terminals. Terminals were very similar to the ATMs. Each Terminal was nothing more than a PC packaged into shiny armor and touch screen.</p><p>One day, the company decided to use smart cards to track users. I had to write a new smart-card-feature for the app. We were not beginners; we had some experience. On a daily basis, we worked with usb, rs232, rfid, nfc, ccTalk, SAS, AT, ICT, custom-made in-house PCBs, and all other kinds of hardware devices. Working on a smart card device feature looked like a task that you assign to yourself when you want to take a rest from the <em>real tasks</em>.</p><p>To accomplish the task, we received a low-budget, no-name, plug-n-play card reader, a few blank PVC smart cards, and SDK. There was no manual, or how-to-guide, or set of instructions, to tell us where to start, what to expect, to warn us about any danger (!), to tell us what type of smart cards are required (there are tons of different card types on the market), to provide any useful information — nothing.</p><p>Communication with the supplier was poor, practically useless. For unknown reasons, the company insisted that we should proceed with that specific supplier.</p><h3>Wrong turn</h3><p>Soon, we realized SDK is not the real SDK. SDK contained two files only. One file was <a href="https://www.computerhope.com/jargon/e/execfile.htm">executable </a>(.exe) file, and it was a demo app with one purpose only — to show us that the device and card work as expected.</p><p>The other file was <a href="https://docs.microsoft.com/en-us/troubleshoot/windows-client/deployment/dynamic-link-library">dynamic-link library</a> (.dll) file. We were supposed that the magic was taking place inside .dll file, and that we were supposed to import and use .dll file inside our app.</p><p>Although there were so many questions, it seemed like Demo App and the card reader device could communicate. With utilities like <a href="https://docs.microsoft.com/en-us/cpp/build/reference/dumpbin-reference?view=msvc-170">dumpbin </a>and <a href="https://www.dependencywalker.com/">Dependency Walker</a>, we figured out the definitions of <a href="https://docs.microsoft.com/en-us/cpp/build/exporting-from-a-dll-using-declspec-dllexport?view=msvc-170">exported </a>.dll functions. The names of the functions were almost identical to the labels on the buttons inside Demo App. It was a good sign — we could associate .dll function with a specific part of demo app.</p><p>When we tried to <a href="https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/interop/interoperability-overview">import</a> .dll into our source code — nothing worked as expected. Actually, we could not establish any communication between our code and .dll. No matter what approach we tried, and we tried a lot of them, the result was always bad. We even tried to <a href="https://www.techtarget.com/whatis/definition/decompile">decompile</a> both Demo App and .dll, hoping that we could reuse the decompiled code, but both files were <a href="https://en.wikipedia.org/wiki/Obfuscation_(software)">obfusacted</a>, so the decompiled version was not very useful.</p><h3>Doom</h3><p>Accidentally, we found out that cards were Siemens SLE4442 EEPROM cards. SLE4442 has a security mechanism (PSC Verification), and improper use of the card can block the card for further usage. Later on, we would found out that we blocked some cards.</p><p>Also, we found out and that Demo App doesn’t work if <a href="https://docs.microsoft.com/en-us/windows/security/identity-protection/smart-cards/smart-card-smart-cards-for-windows-service"><em>Smart Cards for Windows Service</em></a><em> (SCWS) </em>is enabled. This fact was a huge surprise. SCWS is the core part of the OS. It manages all interactions between smart card devices and OS. Just by using common sense one would expect that Smart Card Service has to be enabled when interacting with a smart card reader. Anyway, we concluded that this illogical fact means that our card reader probably does not run on top of <a href="https://pcscworkgroup.com/">PC/SC</a> Workgroup Specifications.</p><p>Days passed by. Frustration was at its peak. We were almost beaten, without ideas on how to proceed. Only thing that kept us trying was the fact that Demo App could interact with the card reader device.</p><h3>Insidious App</h3><p>In the end, we tried to eavesdrop on communication between Demo App and .dll file. The idea was to record data sent from Demo App, and then to sent the same data from our source code.</p><p>The technique for eavesdropping is called <a href="https://medium.com/geekculture/basic-windows-api-hooking-acb8d275e9b8">API Hooking</a>. The concept is comparable to <a href="https://en.wikipedia.org/wiki/Man-in-the-middle_attack">man-in-the-middle attack</a> concept. API Hooking is often used by antivirus softwares, gaming cheaters, malicious softwares, etc. It’s a close relative to <a href="https://www.upguard.com/blog/dll-hijacking">dll hijacking </a>technique.</p><p>Eavesdropping created even more question marks above our heads. There was no communication between Demo App and .dll file. The magic was happening inside Demo App. DLL file was purposeless. The whole time we were trying to do the impossible.</p><h3>Revelation</h3><p>In the end, we discovered that Demo App communicated with some of the core OS system files (it was a long time ago, cannot recall exactly). Although it was a long and painful process, we recorded and reconstructed all data packets sent from Demo App. We put those reconstructed messages to our source code and everything worked fine.</p><h3>Happy ending</h3><p>It took a few weeks just to establish communication between our source code and the card reader device, but eventually the feature went to the production environment. Many, many years later, it still runs on Terminals all across the globe.</p><h3>Part 2</h3><p>Two-three years later, the company created a new terminal, aka Terminal 2. Terminal 2 use SLE5542 cards instead of SLE4442. A small software upgrade was required to run SLE5542 cards. We received the same package as before: a card reader, blank PVCs cards and SDK (Demo App + dll). Version of the .dll was incremented from 1.0 to 1.1 — it was not the same file as before.</p><p>Out of curiosity I tried to import new .dll to source code and to try it. It took me less than a 3 minutes to prepare everything. New .dll worked like a charm.</p><h3>Final word</h3><p>This is an example of HOW NOT TO APPROACH THE TASK.</p><p>Although this journey was unique, although some <em>hardcore</em> approaches were taken, it shows how poor communication, low-budget, personal interests can and will lead to overcomplicated solutions, enormous waste of time and resources, immense frustrations, and, in most cases, poor results.</p><p>Instead of <em>engineering</em>, we were <em>reverse-engineering</em>. There are use cases where reverse-engineering applies. However, if you need to add a simple new feature to the existing codebase, in most cases, that is not a place for reverse-engineering.</p><p><strong>…</strong></p><p><em>Ministry of Programming is a supercharged startup studio specialized in building startups and new products💡 We were voted in the top 1000 fastest growing companies in Europe by Financial Times. Twice.</em></p><p><em>We offer product management, design, development, and investment services to support entrepreneurs and startups towards product success.</em></p><p><em>Building your next startup? We would love to hear more. If you want to work with us on your startup feel free to reach out at — </em><a href="https://ministryofprogramming.com/contact/"><em>https://ministryofprogramming.com/contact/</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1eef6506d2e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mop-developers/how-man-in-the-middle-saved-the-day-f1eef6506d2e">How man-in-the-middle saved the day</a> was originally published in <a href="https://medium.com/mop-developers">Ministry of Programming — Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>