<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ángel Berhó on Medium]]></title>
        <description><![CDATA[Stories by Ángel Berhó on Medium]]></description>
        <link>https://medium.com/@bergraan?source=rss-2fa9776482ea------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LCCnCoPntd27p7EomZAVRQ@2x.jpeg</url>
            <title>Stories by Ángel Berhó on Medium</title>
            <link>https://medium.com/@bergraan?source=rss-2fa9776482ea------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 14:10:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bergraan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[“Maquinillo” as a service]]></title>
            <link>https://medium.com/@bergraan/maquinillo-as-a-service-ba124ff7c53?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba124ff7c53</guid>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[efficiency]]></category>
            <category><![CDATA[engineering]]></category>
            <category><![CDATA[tools]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Tue, 23 May 2023 07:02:22 GMT</pubDate>
            <atom:updated>2023-05-23T07:02:22.723Z</atom:updated>
            <content:encoded><![CDATA[<h4>The hammer you need at any time</h4><p>Hi dear Reader, after a long time without writing a Medium because of lack of time or motivation to write but here I am again to change it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rHw9P8GLVkuUg98f" /><figcaption>Photo by <a href="https://unsplash.com/@mphotographym?utm_source=medium&amp;utm_medium=referral">Moritz Mentges</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>If you aren’t Spanish, maybe “Maquinillo” could be a strange word. There isn’t a translation for this word because It’s an invented word in our business doing references script automation of some input to some output but not for production, which is going to be executed once or twice on a terminal by us and allow us to save time in the short term.</p><p>These kinds of scripts should not be the normal data flow. You should use it for:</p><ul><li><strong>Migrations: </strong>adding a new field in the database and it needs a population before making this field mandatory.</li><li><strong>Batch actions: </strong>perhaps, you didn’t create a batch action and need to create/update a bunch of some stuff. So, the best way is to do it in a CSV process and call an API.</li><li><strong>Export data </strong>that can’t be accessed by the User portal.</li><li><strong>Clean data </strong>and so on.</li></ul><p>As you see, programming helps you with a lot of boring and recurring tasks.</p><p>It seems very legit, but what happens when the “Maquinillos” are the normal flow for a lot of operations?</p><ol><li>You are a <strong>bottleneck.</strong></li><li>These scripts are knowledge that you got in the time you are working. If no one knows these scripts, your business is gaining time in the short term but losing it in the long term.</li><li>The more time you run it, the more likely you are to <strong>fail</strong>. You are in the ocean, don’t swim more than you need.</li></ol><p>We should detect these casuistics and make them known to all the members of our team in order to produce such a flow that anyone can use, for example, an API or interface.</p><p>So, we know when should use them and what are the caveats for these scripts, but how we can create them?</p><p>I use Python for my “Maquinillos”, and also some SQL to export the data as CSV. I feel very comfortable with this language programming and it’s very easy to use but if you don’t know Python or you feel comfortable with another language you don’t need to learn it to create it.</p><p>First, I recommend you always create a new project with the name of the use case. Maybe, the script going to be lower for thinking in a new project but if you open your directory, it’s going to be easy to identify it. In addition, it will help you to know what you need to do.</p><h3>Data</h3><p>This “Maquinillo” has the purpose of <strong>populating/moving/transforming</strong> from A to B, but in my case, I always add two more outputs.</p><h4>Inputs</h4><p>I recommend you always a static source, for example, download a CSV with the target data. Why? because if anything goes wrong in your script, and believe me, you will need to adjust the script to the real output, you can revert it.</p><p>Also, you should use a small file of 5 lines and see the result on the first try.</p><h4><strong>Outputs</strong></h4><p>The output is obvious, the target data is <strong>populated/moved/transformed. </strong>It could be applied to an API, Database, file, etc.</p><p>Another output that I learn in my path to <em>“Maquinillo” as a service</em> is to write always the errors in a file. It’s very important to annotate the errors by line number, ID or whenever you need to identify the data to debug the corner cases.</p><p>And of course, use a logger for reading the terminal and know what is processing or what step gives an error. Some of the interest logs that you will need are the line number/ID before the operations that you need to execute and any label to recognize easy the trace, also the total number of items to process and/or processed. For example:</p><pre>[Info] Items 0/100<br>[Info] Reading rule id 123<br>[Info] Cleaning<br>[Info] Transforming<br>[Error] Error at exporting because the file is blocked, &lt;Some interest params&gt;<br>[Info] Items 1/100<br>[Info] Reading rule id 124<br>[Info] Cleaning<br>[Info] Transforming<br>[Info] Exporting<br>[Info] Finished operation</pre><h3>Execution</h3><p>“Maquinillo” is more than a script. It’s a tool that allows you to make your work easy but it does not pass any tests or revisions. That’s why every try is very important.</p><p>The best suggestions that I can give you are:</p><ul><li><strong>Don’t apply</strong> the operation <strong>until you are secure</strong> with your script. Remember you can break the production server. You are skipping many flows to gain time, make it worth it.</li><li><strong>Notice to your team</strong> that you going to execute the script unless the script isn’t dangerous. For example, it is exported in a file and does not affect to any database.</li><li><strong>Execute it</strong> <strong>in a range of time that allows you to revert it</strong>. Don’t execute it at the end of your and the team&#39;s working time.</li><li><strong>Test it</strong> on dev with a <strong>few records </strong>and not the whole data set.</li><li><strong>Log everything. </strong>all the data you log will bring you superpowers to debug corner cases.</li></ul><p><strong>Bonus suggestions</strong>, if you aren’t very comfortable, make tests of the code that going to be executed that returns something, the entry point going to mutate the data and does not need to be tested with unit tests. For example, read the data, the transformation, or export operation.</p><h3>Process</h3><p>Always, exist an entry point, this going to be configured with the input and the output. It’s just like an ETL (Extract, Transform and Load) but for a very specific case.</p><pre>def execute_maquinillo(input_path: str, output_path: str) -&gt; None:<br>  # Read input<br>  # Transform data<br>  # Export result/Apply data<br>  pass<br></pre><p>Use constants for settings. For example, the directory of the CSV, the API URL, etc… You will see that it going to be fast and clean to see what are you doing and never modify on code unless is really need it.</p><pre>BASE_PATH = os.path.dirname(__name__)<br>CSV_PATH = os.path.join(BASE_PATH, &quot;csvs&quot;)<br><br>INPUT_FILE = os.path.join(CSV_PATH, &quot;some.csv&quot;)<br>OUTPUT_FILE = os.path.join(CSV_PATH, &quot;errors.csv&quot;)<br><br>SOME_API_SETTING = os.environ.get(&quot;SOME_API_SETTING&quot;)<br>...<br><br>execute_maquinillo(INPUT_FILE, OUTPUT_FILE)</pre><p>Use arguments each run you can change anything that allows you to use different files and parallelize work.</p><p><strong>In conclusion</strong>, You are trying to gain some time in the short term by skipping the defined flow or creating it, Maquinillo is a powerful tool for that purpose and probably someone will be thankful to you for that when the data has a large volume and the task going to be tedious, but remember to note these <em>“anomalous cases” </em>because it can be part of your new flow process. And as someone once said: <em>With Great Power Comes Great Responsibility. </em>Thanks, Uncle Ben.</p><p><strong>Thanks for reading</strong>; I hope you like it! I’d love to hear your thoughts and if you used any <strong>“Maquinillo”</strong>.</p><p>Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba124ff7c53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Measure your services]]></title>
            <link>https://medium.com/@bergraan/measure-your-services-6f3ead4ad6f4?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f3ead4ad6f4</guid>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[api-services]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Tue, 10 Jan 2023 11:37:27 GMT</pubDate>
            <atom:updated>2023-01-10T11:37:27.274Z</atom:updated>
            <content:encoded><![CDATA[<h4>Observability part 2: Metrics</h4><p>All services or new features that we have built should be measurable in any way because it will give us feedback on, if that implementation has a good impact or whether it is working well by the number of good vs bad executions and the time it takes to be executed.</p><p>Perhaps you are asking why we should measure our services, and the answer it’s in the production environment we haven’t access to join the machine because of security policies, permissions or it is just a Kubernetes container. Also, if we have permission to into the machine, we shouldn’t do it but we won’t cover this topic.</p><h4>What’s a metric?</h4><p>A <strong>metric</strong> is an <strong>aggregation</strong> from different <strong>events</strong> sent from our app/service/script to a timestamp database and displayed by a dashboard.</p><p>These events should be formatted with <em>&lt;event type&gt; </em>that it’s an identifier that let us observe by a number that could be an accumulator, time delta, or gauge. For example:</p><ul><li><strong>total_requests</strong></li><li><strong>requests_time_summary</strong></li><li><strong>requests_size_summary</strong></li><li><strong>&lt;third_api_service&gt;_time_summary</strong></li></ul><p><strong>Labels</strong> that contain dynamic information about the event fired such as requests method, response status code, etc, and the <strong>value to be measured</strong> over time. These labels should contain data that could be repeated, for example, HTTP method, status code, or path.</p><p>We should create this type of event to measure what interests us depending on our service, this allows us to see if we have created a good architecture or if the service is failing or whether the improvement was good.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8-AspzV4APa2khp2" /><figcaption>Photo by <a href="https://unsplash.com/@lukechesser?utm_source=medium&amp;utm_medium=referral">Luke Chesser</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>As I wrote in the <a href="https://medium.com/@bergraan/light-in-the-darkness-of-services-and-business-ba16b55f476">Light in the darkness of services</a>, if we check the dashboards twice a day we can react to fix the problem and decrease the WTFs metric from our users.</p><p>There are 2 types of tools to send and generate these events, each one has its own applications and we can’t say that one is better than the other:</p><h3><strong>Push</strong></h3><p>It is called push because it’s our service that has the responsibility to send these events to the controller who will listen to and insert them into the timestamp database.</p><h4><strong>Pros</strong></h4><ul><li>It can send events from any service.</li><li>It usually sends them by UDP to save resources.</li></ul><h4><strong>Cons</strong></h4><ul><li>If our service is very concurrent it will DDOS to the Metrics controller and we can’t control the peak of concurrent with Kubernetes services because Kubernetes needs some time to scale up.</li><li>Possible package loss by UDP and we won’t know it.</li><li>A low number of events to send because we can lose the packages.</li></ul><h4>Applications</h4><ul><li><strong>Serverless</strong> because as long as the service is running it will be maintained, but when the services stop receiving executions it will flush all resources to save costs.</li><li>Services with <strong>stable</strong> concurrency.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*6YeECh_rRSMz2zKIEdDh1g.png" /><figcaption>Dataflow push events</figcaption></figure><h3>Pull</h3><p>Unlike the Push type, each service will store the events in the machine/container and the controller will make an HTTP request to discover the new events. You have to configure a file with rules to crawl this information and the controller will make the rest.</p><h4>Pros</h4><ul><li>Bulk events send without packet loss.</li><li>No need to scale the controller service because this one is responsible of get the information from our service.</li></ul><h4>Cons</h4><ul><li><strong>No</strong> supported <strong>serverless or services </strong>that don’t share with <strong>API</strong>.</li><li><strong>Need</strong> an <strong>HTTP</strong> request to extract the data.</li><li>Near real-time, each extraction will do per X time.</li></ul><p>Django generates an event in the middleware, making a request.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0ac30a495d428c574d9ad3b6040c0053/href">https://medium.com/media/0ac30a495d428c574d9ad3b6040c0053/href</a></iframe><h4>Applications</h4><ul><li>APIs with high concurrency.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/601/1*mZXV_HMOxqlFbqZG8us-_g.png" /><figcaption>Dataflow with pull events</figcaption></figure><p>FastAPI generates an event on middleware:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9696008888eb7b7c21171649a804745c/href">https://medium.com/media/9696008888eb7b7c21171649a804745c/href</a></iframe><p>FastAPI generates metrics endpoint to serve the events above:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b19386e823936b833e7330d6bbbd3307/href">https://medium.com/media/b19386e823936b833e7330d6bbbd3307/href</a></iframe><p>We are using Push type in production with <strong>StatsD</strong> and <strong>Fargate</strong> under the hood, over a million daily lambdas execution, and other API services and it’s working well, sometimes we have some Timeouts errors but they lower than 1%.</p><p>In my free time, I started to test Prometheus and it works very well with some APIs that I have built, the only problem that I found and it’s totally solved by them it’s when you have a Kubernetes service where each container has a dynamic IP and you need to request to all nodes because each one contains his events. But it has a controller that makes requests to the cloud provider to resolve the IPs and save them into a file.</p><h4><strong>Display and alerts</strong></h4><p>Finally, with all of these events generated and sent we should display them or make alerts over different metrics. For this purpose, it exists some apps that allow you to create data sources from different databases and use the data, for example, <strong>Grafana</strong> and <strong>Kibana.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Q3fJ0Md_hLo6Mt9xDc8fA.png" /></figure><p>From the different data sources, Different queries can be configured to display different metrics as shown in the image above.</p><p>The next level is to set up alerts based on different metrics to be notified when the metric you set is reached and the provider will do so through different channels such as Slack, email, etc.</p><p><strong>In conclusion</strong>, these tools will provide you with a way to measure your services, Applications such as databases or Nginx are using through plugins. I discovered them in 2022 and I am happy with them because it allows me to know the impact of the features and decrease the metric number of WTFs.</p><p><strong>Thanks for reading</strong>; I hope you like it! I’d love to hear your thoughts and how you measure your services or if you&#39;re going to start to look at these tools.</p><p>Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f3ead4ad6f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How is my remote working environment?]]></title>
            <link>https://medium.com/@bergraan/how-is-my-remote-work-environment-771dd2286c8d?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/771dd2286c8d</guid>
            <category><![CDATA[lifestyle]]></category>
            <category><![CDATA[remote-working]]></category>
            <category><![CDATA[virtual-reality]]></category>
            <category><![CDATA[working-environment]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Sat, 24 Dec 2022 11:01:13 GMT</pubDate>
            <atom:updated>2022-12-24T11:05:01.525Z</atom:updated>
            <content:encoded><![CDATA[<h4>How the setup can improve your performance</h4><p>Hello beloved readers, I have written this post to share how the pandemic and remote affect our work and how I wanna evolve my work environment into Virtual reality or Augmented reality but we will talk about this topic at the end of the post.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bs17zbQ8lpiNn_S1" /><figcaption>Photo by <a href="https://unsplash.com/@thaliatran?utm_source=medium&amp;utm_medium=referral">Thalia Tran</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Before the pandemic, for me, it was normal to work from a laptop, 2 screens and a comfortable desk, I spent my money on something that I wanted to promote in my life because I like to study and develop my own projects. You know for engineers/programmers remote work is quite easy to execute in our diary life because we don’t need to arrange meetings with customers and we do all our work from a laptop.</p><p>With the covid pandemic in the middle, many people had to invest in their environment to work in their homes because there wasn’t a choice to work in the office or coworking.</p><p>I saw at my previous job how much they had to spend on a laptop because they had a computer tower. Or people who didn’t have remote days had to buy a display, chair or even a desk because they didn’t need it before because they were always in the office and now suffer Backache.</p><p>Perhaps you were one of these people, but this post is not to blame anyone. I was surprised because, as I said before, in my free time I liked to study or develop my own projects and I spent money in a comfortable environment, 2 screens, an ergonomic chair and a large table for the screens, a keyboard and space to write notes with paper. I was the nerd when I would activate my webcam in meetings, I’ve always heard a comment about my gaming chair.</p><p>That was 2 years ago, but for a long time, I was thinking about how I could evolve my environment. How could I take advantage of my position working remotely and not be forced to go to the office and have the possibility to work from anywhere but in the right way. But what do I mean by this statement?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5I8zZ3365gMWb3iC" /><figcaption>Photo by <a href="https://unsplash.com/@instagramfotografin?utm_source=medium&amp;utm_medium=referral">Peggy Anke</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Remote work gets people on LinkedIn talking about working at the beach or with their beloved pets, etc. I don’t think it’s a revolution, but I think we should take advantage of this wave to continue promoting this thought since it is not necessary to go back and waste time in traffic jams or make food for the next day when there is already technology that makes us communicate practically as if we were in the office such as Slack, Google meet or Teams.</p><p>If you want to travel while you work you probably are worried because working on a 13&#39; or 15&#39; screen for a lot of hours can help you to hate your environment and come back to the office for getting a large screen or just a second display to be more productive with your tasks.</p><p>On the other hand, when you work from your home, It’s very easy to get distracted by many things because you have things that you did not have before in the office, for example, Your phone may ring and when you didn’t pick it up before, now you think about doing it. Also, Your neighbors may be renovating their house, your dog barking, or your children hanging around the house.</p><p>So how can this be fixed, or at least a little bit? A few years ago, Meta (formerly Facebook), Microsoft, and other brands started developing Virtual Reality (VR) and Augmented Reality (AR). Maybe, you can read all about the Metaverse where you can do anything, like in the movie Matrix. I liked the idea of working anywhere but without distractions, so I started thinking about my ideal environment. That made me follow the VR thing and see how it evolved.</p><p>Then, I saw how you set up your headset and you can forget about your “Reality” because you are in another reality while working or playing, at first, Meta started developing Oculus for the purpose of gaming, but now you can work with apps that link to your computer and add a lot of screens without investing in it. just with one click, you can add a new screen.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F1TLC2tdImZ0%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D1TLC2tdImZ0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F1TLC2tdImZ0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff7f324ffdea11b246f0315a2e4a84d2/href">https://medium.com/media/ff7f324ffdea11b246f0315a2e4a84d2/href</a></iframe><p>This type of technology isolates you from distractions and you can travel with just a laptop and glasses. All these features are good, but there are some drawbacks that may put you off, for example:</p><ul><li><strong>Hardware is expensive, </strong>if you start from scratch you will probably have to spend a lot of money.</li><li><strong>Low Autonomy, </strong>Headsets are very powerful to manage all sensors and display the environment, they can give you 2 hours up and 2 hours more if you buy an elite strap with an external battery or plug it into the charger with a cable.</li><li>Some people can’t stay long playing with them because they get dizzy.</li><li><strong>Difficult to set up</strong> at the first.</li><li><strong>Good internet, </strong>if you want to link with the computer it will need a large bandwidth with router 5G or Wifi direct. At this moment is not possible to do it from Meta Link (wire) because there isn’t API for it.</li></ul><p>Of course, brands strive to improve them and try to make them accessible to all customers. For now, I’m testing how I can do it in the best way and let me work anywhere my job allows me. My actual working environment at the Virtual beach 😎</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LEh58vJVkVwYooirpGdx7g.png" /><figcaption>Working environment with 4 screens</figcaption></figure><p>The image has opacity but trust me you can work there, at this moment I&#39;m writing the post with the headset.</p><p>This is the evolution of my new environment and the new comments about “<em>the nerd and the metaverse”</em>. You might think it’s not for you. I can only suggest to you try it if you can do it or make your work environment the most comfortable if you need to sit in the same place for long hours.</p><p><strong>Thanks for reading</strong>; I hope you like it! I’d love to hear your thoughts and how you work, your preferences, or your setup.</p><p>Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=771dd2286c8d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Light in the darkness of services and business]]></title>
            <link>https://medium.com/@bergraan/light-in-the-darkness-of-services-and-business-ba16b55f476?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba16b55f476</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[metrics-and-analytics]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Mon, 19 Dec 2022 17:49:31 GMT</pubDate>
            <atom:updated>2022-12-19T17:49:31.935Z</atom:updated>
            <content:encoded><![CDATA[<h3>Light in the darkness of services</h3><h4>How can observability help you in your diary life?</h4><p>Hi folks! Today, I’m going to talk about observability in services. I probably won’t resolve all your problems but maybe I can respond: “Why should I start to apply observability to your projects?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wx9E5QBzh-sL6pTp" /><figcaption>Photo by <a href="https://unsplash.com/@spacex?utm_source=medium&amp;utm_medium=referral">SpaceX</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Before, start speaking about this topic, I would like to say this Medium is for all that are worried or want to know more about it.</p><p>Let’s imagine we have created an API service that will be consumed by our front end. In the early, If your company is a little company or giving a “Hello world” to others, you know it will be a few requests for this service, but when it begins to grow up there are some metrics that start to grow also like: “<em>The service </em>is going very” or “<em>The service is not working as expected</em>”. These errors could be reported by our loved product team, but you are still developing new features and “maintaining” the service. Spoiler, you’re just <strong>developing</strong> <strong>new features</strong>.</p><p>In the new environments, when you deploy a service to production, you probably did it in Kubernetes, or a cloud like AWS, Google Cloud, or others. They do real magic because the service scale up and you don’t need to know when, you give the money and forget it. This is a fatal error for different reasons, but the main one is: “<em>Do you know why is scaling up? Can you prove that your service is slow?” </em>or,<em> c</em>an you respond to your beloved product manager with these questions?:</p><ol><li>Your request to <strong>DB</strong> or <strong>third-party API</strong>, Are they the bottleneck to your service?</li><li>Are you requesting a lot of requests to load the data that you need for your services?</li><li>Is your service healthy?</li><li>… And so on.</li></ol><p>For this challenge, we should learn about how our service is performing, and for that task, <strong>we need data! </strong>and there are 2 types:</p><p><strong>Metrics</strong> are a way of measuring how your service is performing with our infrastructure and connections with others. It doesn’t matter if the service is an API or Lambda or other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6v6ejDIxikFP__kykoyZgQ.png" /><figcaption>Grafana dashboard</figcaption></figure><p>For example, if we have an API, some metrics should matter; the <strong>Number of executions</strong> (Total and per endpoint), the Average and highest percentile of the <strong>time to be executed</strong>, and the <strong>Number of success and error executions</strong>.</p><p>This will help you in the first instance to know more when your services are working or not much. Also, if your service is calling the external client, you can measure it in the same way and it will let you know why your service is slow or if the third party is failing.</p><p>For this topic there are some tools for that, in the first instance to store metrics, we can find: Prometheus, influshDB, etc… and deploy it in the Grafana or Kibana dashboards.</p><p>On the other hand, we can measure our business logic by <strong>Events</strong>, this way of measuring our application will help our data team to investigate, explore and propose new projects for the business but also to take advantage of it and implement AI algorithms. Imagine that the service is in charge of renting hotel rooms. It may be interesting to know the number of hotels that were partners with us in the last month, and the number of clients who rented a room to know if there is a special event such as a concert that allows us to increase offers or try to have enough stock for the Black Friday.</p><p>For this purpose, we can find Big databases that act like a data lake and will allow you to ask how your business is doing, such as AWS Athena and Google BigQuery. And there are a lot of tools for displaying this information as Metabase, Power BI, and others.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/0*xnzMoLg3dpC496gS.png" /><figcaption>Photo by Metabase</figcaption></figure><p>In conclusion, to answer the question at the beginning, we have to know in general terms what our application is doing to improve it and obtain all the information possible to take advantage of it to open different branches of business with new ideas.</p><p>Thanks for reading; I hope you like it! I’d love to hear your thoughts and experiences about your builds.</p><p>Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba16b55f476" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rust: Singleton Application]]></title>
            <link>https://medium.com/better-programming/rust-singleton-application-db2df027afa1?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/db2df027afa1</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Tue, 17 May 2022 20:57:46 GMT</pubDate>
            <atom:updated>2022-05-17T21:02:17.505Z</atom:updated>
            <content:encoded><![CDATA[<h4>Borrow checker vs design patterns</h4><p>Hi everyone. Let’s talk about one of Rust&#39;s concepts that make Rust developers go crazy: the Borrow checker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*pCBgZgFrh5NL6-5i" /><figcaption>Foto de <a href="https://unsplash.com/@lazycreekimages?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Michael Dziedzic</a> en <a href="http://localhost:3000/es/s/fotos/design-pattern-language?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>But first, let me explain what this is and why you should be interested in this topic if you want to start to develop in Rust.</p><p>In other compiled languages, such as C++, you must control memory, i.e., allocate and free dynamic memory. In others like Java, you don’t face this problem because a garbage collector is running and listening for unused memory to free it in the background.</p><p>Rust makes it “simpler” or makes it seem simpler, because when a scope is terminated, Rust frees it. Also, when you pass the value to a function/method (another scope), this scope gets the variable property, and when its scope terminates, this memory is freed again. Let’s see an example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/93dd2dd7187c59bf8588581c0067c7e4/href">https://medium.com/media/93dd2dd7187c59bf8588581c0067c7e4/href</a></iframe><p>Therefore, if we want to reuse some_string in these functions, we have to switch to get a reference instead of the variable&#39;s value in the print function.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cda73e15fd7ba02e1465932d77fabe57/href">https://medium.com/media/cda73e15fd7ba02e1465932d77fabe57/href</a></iframe><p>Or return the value again to not free it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f6a3e100e43fab82948b3c9f4ecca05e/href">https://medium.com/media/f6a3e100e43fab82948b3c9f4ecca05e/href</a></iframe><p>There are other rules with the borrow checker that can raise an error. For example, when passing a reference to a struct constructor, the borrow checker will raise an error because it doesn’t know how long the struct or the variable you pass as a reference can live, so the compiler will force you to use lifetime. But, beware, this is a trap.</p><blockquote>Borrow checker is the guard to avoid memory leaks</blockquote><p>We can say the borrow checker is the guard to avoid memory leaks that make you debug for hours to find out the occasional bug.</p><h3>Design Patterns</h3><p>There are some design patterns, such as Singleton, that the borrow checker might hinder. This is because, in the Singleton strategy, you must store an instance of the structure as static in the structure.</p><p><em>Singleton is responsible for storing a global instance of itself and being reused by the entire application to store global variables such as configurations or connections to a service.</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8073951d15838c519f27dc841e3a1653/href">https://medium.com/media/8073951d15838c519f27dc841e3a1653/href</a></iframe><p>In theory, the singleton works like this. We create a class/struct with an attribute to itself and a method that creates an instance of itself and returns it.</p><p>Spoiler: Rust doesn’t let you create this for two reasons:</p><ul><li>You can’t use an attribute from a struct without instantiating.</li><li>The borrow checker won’t let you store an instance of itself because it detects it as a memory leak.</li></ul><p>That’s why Rust core developers design some tools for this.</p><h3><strong>Reference Counted (aka Rc) struct</strong></h3><p>This structure allows you to share references between different domains by means of a weak reference. How does it work?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/45c7b9942338f6975ae7dfc6e184d46e/href">https://medium.com/media/45c7b9942338f6975ae7dfc6e184d46e/href</a></iframe><p>This makes the code compile because the Rc structure creates a weak reference as an attribute that tells the compiler that it doesn’t need to check at compile time. The problem is we aren’t using this attribute after creating it. So, in the next call, it will create it again.</p><p>However, we can use this tool to structure data as a dynamic list, but not as a Singleton.</p><p><em>This only works in a local thread. If you need more threads, I recommend you use Arc. This trait implements the Clone trait to clone references with threadsafe.</em></p><p><strong>Mutex</strong></p><p>It will store our value as mutable but will block the resource until someone leaves the reference to this value. For example, if we have a variable of type u8, it will make it mutable even though it is a primitive type.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c2f988956b96f0dee84ab0ef90dad6b7/href">https://medium.com/media/c2f988956b96f0dee84ab0ef90dad6b7/href</a></iframe><p>So, why is it still not getting the actual value? Well, in the get_instance we are instantiating the Singleton structure each time we call it and not using our actual instance attribute. So, we have to get from any global state.</p><p>What’s the problem? Static variables can’t be mutable because Rust believes you can create race conditions, and so it forces you to use unsafe scope.</p><p><strong>thread_local!</strong></p><p>This macro allows the creation of static variables without a non-constant value treatment. Here’s an example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/67fe2766e8151ed06572c47e08fe96dc/href">https://medium.com/media/67fe2766e8151ed06572c47e08fe96dc/href</a></iframe><pre>#[derive(Default)]</pre><p>This derive makes our struct implement the Default trait. The trait implements the primary and non-primary default values of Rust. So, we don’t have to worry about defaults now.</p><pre>thread_local! {<br> static SINGLETON_POOL: Arc&lt;Singleton&gt; = Arc::new(Default::default());<br>}</pre><p>Creating a static default value of our Singleton. In addition, we have changed Rc to Arc (Atomic Reference Count), which allows us to clone the reference.</p><pre>pub fn get_instance() -&gt; Arc&lt;Singleton&gt; {<br> SINGLETON_POOL.with(|singleton_pool| singleton_pool.clone())<br> }</pre><p>The get_instance method now returns a global state of the Singleton. And voilà<em>, </em>we got our singleton working.</p><h3>Conclusion</h3><p>It is not a good idea to create a Singleton as other languages do. Perhaps it could be simpler to store the variable in the parent state and share it among the components that need this configuration. For example, the Actix web framework implements a state that will be passed in each request.</p><h3><strong>Resources</strong></h3><ul><li><a href="https://doc.rust-lang.org/book/ch04-02-references-and-borrowing.html">Borrow checker docs</a></li><li><a href="https://blog.logrocket.com/introducing-the-rust-borrow-checker/">Borrow checker blog</a></li><li><a href="https://doc.rust-lang.org/stable/std/rc/struct.Rc.html">Rc docs</a></li><li><a href="https://doc.rust-lang.org/std/sync/struct.Arc.html">Arc docs</a></li><li><a href="https://doc.rust-lang.org/std/sync/struct.Mutex.html">Mutex docs</a></li><li><a href="https://doc.rust-lang.org/std/macro.thread_local.html">thread_local macro docs</a></li><li><a href="https://docs.rs/actix-web/latest/actix_web/struct.HttpRequest.html#method.app_data">Actix web app state reference</a></li></ul><p>Thanks for reading; I hope you like it! I’d love to hear your thoughts and experiences working with design patterns or Rust.</p><p>Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db2df027afa1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/rust-singleton-application-db2df027afa1">Rust: Singleton Application</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Design patterns — your software factory]]></title>
            <link>https://medium.com/@bergraan/design-patterns-your-software-factory-8badddbc97?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/8badddbc97</guid>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Wed, 04 May 2022 06:08:37 GMT</pubDate>
            <atom:updated>2022-05-04T06:08:37.775Z</atom:updated>
            <content:encoded><![CDATA[<h3>Design patterns — your software factory</h3><h4>Adapt your software to your needs</h4><p>Hi everyone! We met again to enjoy this new post. Get your favourite drink, and dive into this reading with me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*vx4bDHB9pRKOMDtv" /><figcaption>Photo by <a href="https://unsplash.com/@windows?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Windows</a> on <a href="https://unsplash.com/?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>For this story, let’s talk about design patterns. We did it in my previous post about “<em>Hexagonal architecture</em>”, where I exposed some problems with the Django framework that makes you couple your business logic with the data and representation layers.</p><p><a href="https://medium.com/@bergraan/hexagonal-architecture-catch-em-all-496cc4a88734">Hexagonal architecture: catch ‘em all</a></p><h4>What are design patterns?</h4><p>They are <strong>solutions</strong> to <strong>well-known problems</strong> that all developers use in their vocabulary to refer to any part of the software architecture. Sometimes they are usually beneficial for the application but not always, they just resolve generic problems but if you need to resolve a specific problem, maybe you can adapt it to solve it.</p><p>There are some types of design patterns:</p><ul><li><strong>Creational: </strong>provide some object creation mechanisms to increase the flexibility to reuse existing code. It’s a functionality to return a piece of software that can interact if changes in the future.</li><li><strong>Structural: </strong>explain how entities and models can interact between functionality.</li><li><strong>Behavioural: </strong>create a way to interact between functionalities.</li></ul><p>These types of design patterns aren’t incompatible, they complement each other and sometimes also are similar changing the context.</p><p>Django framework loves design patterns. For example, you can set up different database connections just with the configuration and finally, when starting your development, models call the manager and these create queries and execute them in the connectors.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c04d21f580cc1d30a04319800d059cf7/href">https://medium.com/media/c04d21f580cc1d30a04319800d059cf7/href</a></iframe><p>But… how are they doing? 🤔 First, they use some design patterns for this casuistic:</p><h4>Adapters</h4><p>It’s a structural pattern that allows us to deal with different problems in the same way in different formats. For example, to get information from API rest, databases or events.</p><p>Adapters help us to change between them without understanding how they get the information. The parent component just wants the data, not the way to get it and the adapter fit well with this purpose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/301/1*76FgypECsVi1eNFOiwtOag.png" /></figure><p>This can be translated as code on the following lines</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f5635072d25f8b5441325c34ac072e0/href">https://medium.com/media/7f5635072d25f8b5441325c34ac072e0/href</a></iframe><h4>Factory Method</h4><p>It’s a creational pattern that allows you to interact with an object in the same way; how? it’s simple, we have to define what pieces we need as a puzzle and the factory who is responsible for returning the specific piece in base an input.</p><p>The main character is the factory entity that is in charge of choosing which part we want, avoiding the need to use ifs or switches in our code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/301/1*wKUNBXBs3UgGvLamLDZp9g.png" /></figure><p>As you can see in this diagram, we have just defined an <strong>Adapter </strong>with get_result operation. This means that all components that use an adapter don’t need to know the specific implementation of the adapter, only the generic functionality.</p><p>The <strong>factory </strong>in Python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b64cdb405797139822078c51fb949b54/href">https://medium.com/media/b64cdb405797139822078c51fb949b54/href</a></iframe><p>With this design, you can use it without understanding what logic is inside the logic adapter, it only matters what we want to use using a configuration.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/deeae8b973f0ee9406a0bfdd992b6019/href">https://medium.com/media/deeae8b973f0ee9406a0bfdd992b6019/href</a></iframe><p>As you can see, we do not care about the source, only about the source response and all of them are homogeneous.</p><p>Then, we can figure out how Django does to translate from his manager system to connect with different databases.</p><p>We can see each database as an adapter and these adapters should have a execute_query which opens the connection with his source and implements the cursor functionality serializing each row with our model. Then, these adapters are:</p><ul><li><strong>SqliteAdapter: </strong>open the SQLite file.</li><li><strong>MysqlAdapter: </strong>open connection with MySQL client to MySQL DB.</li><li><strong>PgAdapter: </strong>open connection with psycopg2 client to Postgres DB.</li><li>Etc…</li></ul><p>This makes the code very flexible to create other adapters or switch between existing and decouple from the parent component.</p><h4>References</h4><ul><li>Design patterns <a href="https://refactoring.guru/design-patterns/">web</a> — by <strong>refactoring.guru</strong></li><li>Django <a href="https://docs.djangoproject.com/en/4.0/ref/databases/">settings</a> docs — <strong>Django</strong></li></ul><p><strong>Thanks for reading, </strong>I<strong> </strong>hope you like it<strong>! </strong>I’d love to hear your thoughts and experiences working with design patterns or without them. Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>. 😃</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8badddbc97" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hexagonal architecture: catch ‘em all]]></title>
            <link>https://medium.com/@bergraan/hexagonal-architecture-catch-em-all-496cc4a88734?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/496cc4a88734</guid>
            <category><![CDATA[django]]></category>
            <category><![CDATA[clean-architecture]]></category>
            <category><![CDATA[hexagonal-architecture]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Mon, 14 Mar 2022 19:43:08 GMT</pubDate>
            <atom:updated>2022-03-28T07:57:29.230Z</atom:updated>
            <content:encoded><![CDATA[<p>Hi reader! Let’s talk about hexagonal architecture. This post is for all those who started in the development world or just want to learn something new about app architectures instead of learning more languages or frameworks. So, before you start to read it, grab your favourite drink and enjoy it 🙂</p><p>If you remember the beginning, your teacher or any getting started post you’ve read on how to start coding, it should start with the stages of developing an application. In case you missed it or simply forgot, let’s take a look at them:</p><ul><li><strong>Conceptualization: </strong>We know what we want but we have to find the answer to the questions, e.g. <em>How it can help in my life? </em>Is it worth developing?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*GzIT4klpfA8J3YlizB9dXQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ameenfahmy?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ameen Fahmy</a> on <a href="https://unsplash.com/s/photos/idea?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><ul><li><strong>Definition</strong>: After solving the conceptualization we have to specify what features the application should have.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VzFSQFKzCBqP5BpCXJuAYg.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@campaign_creators?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Campaign Creators</a> on <a href="https://unsplash.com/s/photos/brainstorming?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><ul><li><strong>Design: </strong>At this stage, it depends on the purpose of the app. If it is for the backend, we must define the structure and data flow. On the other hand, if we want to create a frontend application, we need wireframes of windows.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mkG0YtCzyPQpU9MG0HI79w.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@d_mccullough?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Daniel McCullough</a> on <a href="https://unsplash.com/s/photos/architect-design?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><ul><li><strong>Development: </strong>easy, time-consuming coding of all features, bugs, refactorings, etc.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*GHDLpmatCad_ulP_vdMnxQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@walkator?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Walkator</a> on <a href="https://unsplash.com/s/photos/code?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><ul><li><strong>Publish: </strong>deploy the app for our purposes</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*pnWF54uhru4HSE4WzKTMKQ.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@chesterfordhouse?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Peter Lawrence</a> on <a href="https://unsplash.com/s/photos/valencia?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Hexagonal architecture is part of the design stage, why? conceptualization and definition are part of the business logic. In the design stage, we have to define what we need to build for our features.</p><p>This is the stage that is usually the most underrated. Some people, I can include here. They think about how to build the application with any framework in their mind because it is popular or just because it can make their work easier at the beginning and can save time in development time. For example, <a href="https://www.djangoproject.com/">Django</a>, let’s talk about it later.</p><blockquote>Not all applications can use the same technologies because not all business logics are the same.</blockquote><p>I have learned that each app needs different technologies, for example, if you want to build a CMS you probably want to use <strong>WordPress</strong> or <strong>Wagtail</strong>, in other contexts, you might want to build a fast API to prototype the business with <strong>Node</strong> or <strong>Django</strong> and in the future when the app grows and receives a large number of requests refactor with a language that can handle it, for example, <strong>C++</strong>, <strong>Rust</strong> or <strong>Go</strong>.</p><p>You can build a CLI in different languages, AWS for example did it in Python but they could do it in C++ even though they knew it was better to improve time than performance because it’s not going to scale, they have to improve their performance on the backend where all the people are going to contact them.</p><h3>What is Hexagonal architecture?</h3><p>Hexagonal architecture is a software design in which all the logic is separated into different domains and layers. This architecture can be called “adapts and ports” because it is the parts that are divided.</p><ul><li><strong>Ports:</strong> are the implementations to connect to any service, for example, Database, Cache, FileStorage and Email.</li><li><strong>Adapters: </strong>are the<strong> </strong>devices that will connect to these ports.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*UlPKSPerygg2cVZgl1CFhA.png" /></figure><p>This architecture tries to solve the <strong>D</strong>ependency <strong>I</strong>nversion <strong>P</strong>rinciple (<strong>DIP</strong>). As <strong>Bob Martin</strong> said it must be divided into layers, and these layers are not coupled to each other. What does this mean? if we change any logic in layer <strong>B</strong> that was invoked by layer <strong>A</strong>, it <strong>should not change</strong> or change minimally.</p><p>Suppose we are building a backend API and it was built with a Controller (Port), a Service (Business Logic) and a Repository (Port). If any logic changes inside the repository, the Service and the controller should not change. On the other hand, if we want to change the repository (port) from a memory to a database, the Service should depend on the interface and not on the details, it should always abstract. Our components only have to know our implementations.</p><p><strong>Why Should You Use Ports and Adapters?</strong></p><p>An architect must think about how the application can scale. Over the life of the application, there are some pieces that may change based on business logic, performance or other circumstances. If you don’t think this way, when this happens it will be difficult to make the change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*Fi75S-YVnyGyMJ2fJ1xPqA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@ryoji__iwata?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Ryoji Iwata</a> on <a href="https://unsplash.com/s/photos/puzzle?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Other advantages of dividing the code into different parts are:</p><ul><li><strong>Separation of concepts:</strong> think business logic or domain (DDD).</li><li><strong>Easy testing:</strong> each part only has to test its functionality, which makes it less expensive to create runtime use cases and lines of code (unit tests).</li><li><strong>Easy to change and maintain</strong></li></ul><p>For example, if we want to migrate from Memory to Postgres to store data we just need to create a new layer with this logic and change its invocation to the new one (Repository). To do this, we need to create our Port first.</p><p>For our DBPort, we define the rules to play in our application. We can check in the following example, we have the CRUD operations as abstract because it must not have logic, it is the only definition that each port must have, implement and understand in other layers. In addition, we define a BaseModel that must always be in the models to return the same types.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/71af7855d5b3a447bff5cbf6c7172593/href">https://medium.com/media/71af7855d5b3a447bff5cbf6c7172593/href</a></iframe><p>Now, let’s create our memory DB port. This is the same as DBPort, we want to define the rules for our <strong>MemoryDBPort</strong> to work.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f953bd19b29759b40e6527199caebc6/href">https://medium.com/media/0f953bd19b29759b40e6527199caebc6/href</a></iframe><blockquote>It is very dangerous to use it in production, it is just an example, do not use it.</blockquote><p><strong>MemoryDBPort</strong> overrides all abstract methods that change the table structure. It must raise a common exception and the same input and output because those invoking this repository must catch possible errors and will return a BaseModel. In this example, it is very simple because the only error raised is <strong>IndexError</strong>.</p><p>We have created another Port to implement the <strong>PostgreSQL</strong> database.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0bedee88b722fc51002e9e2e64527a1/href">https://medium.com/media/f0bedee88b722fc51002e9e2e64527a1/href</a></iframe><p>As you can see, this port in the init class will init the client connection to the database, and it will overwrite all the methods to implement the logic as the architect thinks of it in the interface and what our application will understand. So now, it’s very easy to switch between these ports because the dependency is less. In the service, we just have to reference one or the other port. Your logic will not change because the methods are the same.</p><h3><strong>Example with no hexagonal architecture</strong></h3><p>This example has been done purely for learning purposes, not to say that any framework is worse than others. I have worked with Django for a long time and I think it is still very powerful.</p><p>Let’s start with the example, let’s suppose we want to build an application that stores a Pokemon team for a tournament, we don’t need more details than each team has 6 Pokemon we don’t care about tournaments, it’s not our business. Our goals, in this case, are to have a database of Pokemon and teams.</p><p><strong>What are our needs for this?</strong></p><ul><li><strong>Team Service: </strong>You need to verify that the equipment exists, but you have a requirement. The team service has to verify its owner, and that is with a user.</li><li><strong>User authentication: </strong>the owners of teams.</li><li><strong>Pokemon service: </strong>We need to verify that the Pokemon exists to maintain consistency between services.</li></ul><p>For this purpose, we have the following diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2axQs-XcFhPryo9G1dbT7A.png" /></figure><p>Great, we have our <strong>UseCase</strong> and a little mind map of how we can relate all the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*LUUG76TxRrwKCpzWt12ZZg.png" /></figure><p>We chose this flow with Django. Django is a web framework based on MVC architecture, so it provides us, the <strong>models</strong>, the <strong>views</strong>, the <strong>controllers</strong> (not exactly in the controller) and the logic of the <strong>templates</strong>. For this example, we also add Rest-Framework, the library to implement the API logic.</p><p>Django allows you to create an app per domain. App for Django is a module that contains all the business logic. So we have, 3 domains:</p><ul><li><strong>Authentication</strong></li><li><strong>Pokemon</strong></li><li><strong>Teams</strong></li></ul><p>Django as I said is powerful, so it gives us all the auth domain, so we decided to reuse this.</p><p>When we start implementing our data logic, we can see that Django allows us to relate data between different domains. It’s not bad but it’s not good. Why? Imagine we want to change the authentication domain to another port because we need to <strong>migrate</strong> to another <strong>authentication provider</strong> because it offers some features that we don’t want to develop over time and maintain because of the cost. We don’t want to maintain this domain when it is someone else’s thing and they will do it better than us probably.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/069ef5d93b2e574e73c485dfb690ca26/href">https://medium.com/media/069ef5d93b2e574e73c485dfb690ca26/href</a></iframe><p>With this approach, we are relating the <strong>authentication</strong> and the <strong>pokemon</strong> domain in the same database.</p><blockquote>The <strong>ForeignKey</strong> and <strong>ManyToManyField</strong> classes have a lazy relation constructor, which means that Django can resolve and import the dependency automatically, but this is bad because we are coupling another domain’s data model.</blockquote><p><strong>Views Logic</strong> was built with classes, very simple, you just inherit from a generic class and the mixins you want to interact with the API data. GenericViewSet depends on <strong>serializer_class</strong> and Q<strong>ueryset</strong>, and the rest of the framework does its magic (blackbox, we don’t care how), for example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8954cfa832e7e12441513316af693fc/href">https://medium.com/media/e8954cfa832e7e12441513316af693fc/href</a></iframe><p><strong>Problems?</strong></p><p>We are merging ports and data in the same place, if in the future we want to make a CLI with the list function we must keep 2 places in code to do it because there is no service controlling the business logic. On the other hand, using the API but we must deal with the same authentication method when we probably want to do it as admin otherwise. DRY fails.</p><p><strong>Serializers</strong></p><p>Rest-framework serializers have a lot of magic. When you think of a serializer, you only think of serializing and deserializing data. Rest-framework serializers take care of that and more.</p><p>You can pass the query to the serializer and the serializer will execute the queries for you. This means that the <strong>View</strong> and <strong>Data</strong> serialization are controlling the same thing. included, if you want to create/update any object, the serializer will do it for you. This is very convenient at the beginning but a pain in the future.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d325bbbae78a172849324d6cc9b3a52d/href">https://medium.com/media/d325bbbae78a172849324d6cc9b3a52d/href</a></iframe><p>I remember when I was learning FastApi, and I asked the owner how can I implement a database validation. I was doing it in the View because I had no way to do it inside the serializer. His <a href="https://github.com/tiangolo/fastapi/issues/328">response</a> was that I was doing it the right way. Data validation should not have the responsibility of business logic. Great guy 😃</p><p><strong>Testing</strong> starts to get complicated. We can’t unit test just any piece because if we test a serializer, we have to test a tested piece just to test our validation and serialization of data and queries, same for the view, we have to instantiate with a lot of magic that Django provides. Mocks? we have to search we have to mock and do it the right way and it can be tedious.</p><p>The easy way is to test it as <strong>e2e</strong>, what does that mean? we have to instantiate the whole application and read the documentation. Django has a command where you can test all applications or just a domain.</p><blockquote>python manage.py test [domain1, domain2…]</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c25beffac6b1a7da09d8f802a10c1cf/href">https://medium.com/media/3c25beffac6b1a7da09d8f802a10c1cf/href</a></iframe><p>As you can see, we need to test: the case of <strong>no authentication</strong>, the case of <strong>success</strong>, the case of <strong>adding pokemon for another team</strong> and the case of <strong>pokemon does not exist</strong> when we should really just test the business logic from the service and the view. In this test case, we are validating authentication, business logic, data validation and creating our fixtures inside the database. It is not worth it.</p><p>If you want to take a look at this project, you can see it <a href="https://github.com/bergran/pokemon_project_example">here</a>.</p><p>In <strong>conclusion</strong>, you can see that this app is very simple and we have built it in a short time, this is a good approach if the business needs to create an MVP with little budget at the beginning, but if the business scales, our app will make it difficult to continue over time. The app development time will change a lot depending on the business needs, a good architect should deal with it in a simple way.</p><p>In the next part of this series, we will see how we can implement this logic as a hexagonal architecture in different steps.</p><p><strong>Thanks for reading, </strong>I<strong> </strong>hope you like it<strong>! </strong>I’d love to hear your thoughts and experiences on it. Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>. 😃</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=496cc4a88734" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Happens If We Code the Same Algorithm in Python and Rust?]]></title>
            <link>https://medium.com/better-programming/from-pythonic-to-rustacean-this-is-the-way-6ee46ee63033?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ee46ee63033</guid>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Sat, 05 Mar 2022 10:08:22 GMT</pubDate>
            <atom:updated>2022-04-03T18:07:08.171Z</atom:updated>
            <content:encoded><![CDATA[<h4>From Pythonic to Rustacean — is this the way?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qr_PcbEsJFZo4EBq" /><figcaption>Photo by <a href="https://unsplash.com/@jannerboy62?utm_source=medium&amp;utm_medium=referral">Nick Fewings</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Hi again, dear readers! After a long time, I come here to write a post about my first experience with Rust.</p><p>Perhaps you are asking yourself (or not) why about this post. Some weeks ago, I read a post from AWS where they talked and display a graphic with all the language codes comparing the Sustainability of all of them. Link to <a href="https://aws.amazon.com/es/blogs/opensource/sustainability-with-rust/">post</a>. In that article, they talked about how they scaled up his platform and how good was to do it with rust. At that moment I was “Wow, really?”. Some days later I read a Twitter thread where the author argues about this post is not saying all the truth. But I’m not here to talk about that. Anyway, if you want to read it too, here is the <a href="https://twitter.com/_rsc/status/1496352325157457922?s=12">link</a>.</p><p>I‘m a very curious person and a very geek about this topic, so I just started to look up information about Rust. Like the first steps to get started, some GitHub repositories to see how it is the language and I liked it!</p><h3><strong>What is Rust?</strong></h3><p>From his page you can get a definition like “<em>A language empowering everyone<br>to build reliable and efficient software.</em>” but what is really mean?</p><p>Well, Rust is a Compiled language code, this makes our code transforms into machine language, and we could say that our whole process runs are executed as if the hardware get our app and runs it, this should be very fast.</p><p>Rust was created in 2010 by Mozilla, ten years later was created Rust foundation to maintain this language as an open-source language like Python and it&#39;s a very popular language by stack overflow in their surveys for example this in <a href="https://insights.stackoverflow.com/survey/2015">2015</a>.</p><p>What are the main features? when they developed this language they have in their minds security, When the compiler is called this inspects your code to check if there are any memory bugs or bad pointer references.</p><h3><strong>Apps you can build</strong></h3><p>If you think of any language code you probably relate with any type of app, for example, <strong>Swift</strong> for iOS mobile apps, <strong>Python</strong> for Web servers and ML, JS for web development, or <strong>Java</strong> for 3 billion devices.</p><p>But nobody will say JS can make web servers or Python can’t scrape webs because probably someone does. And that’s why Rust can do a lot of types of apps: CLI, web server, web assembly (some browsers can execute rust code), Scraping, Microcontrollers (embedded devices). Best of all, it is always with the best performance and low resources compared with other languages codes.</p><h3><strong>Motivation</strong></h3><p>When I was a student I started with C and Java, but when I met Python, I fell in love. I think is a very good language to start with, it has a great community and it has a lot of powerful libraries which you can learn a lot of how is the software and understand it.</p><p>Finally, when you get the experience and you think you are ready to step up, you can play in the league of compiler languages because you come with design patterns, know how the software works and you only have to add a new variable more, the memory control.</p><p>It’s like a car racing videogame, you can’t start with the max difficulty, you start in the low or medium-level and grow up when you get control. This is very similar.</p><h3><strong>Python vs. Rust: Through a Fibonacci Algorithm</strong></h3><p>As I said before, Python is a great language to start learning to code. It’s an interpreted language, this script needs python on any device that we want to run it. It’s weakly typed and has a very cool syntax called Pythonic.</p><p>Against Rust, we can see the next diagram where Python is executed in any device that has Python installed and Rust just needs to compile the script to translate it into machine language and finally, execute it on any device because the machine can understand it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OQl3jGkWnXSZwFpcRhiKew.jpeg" /></figure><p>Python gives us fast development but it has a big con, his execution is expensive and lower. If we compare the same Fibonacci algorithm we can see a big difference between times.</p><p>To see it in a practical way, I coded the same algorithm into Python and Rust.</p><p>At first, if we take a look at the Python code, we can see that the lines of code are inferior to Rust. This is because the Python core developers design python to be legible. They did a great job.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1bb6c5754345bc66d7a9114aa8cc2086/href">https://medium.com/media/1bb6c5754345bc66d7a9114aa8cc2086/href</a></iframe><p>In Rust code, we close duplicate the lines. Rust divides code blocks with curly braces while in python the blocks are divided by tabs and spaces. Also, we had to import some libraries to manage big numbers, io, etc.</p><p>Python interpreter manages the memory by default but in Rust, you must define which type of variable is it since the max static type of int is u128 (2¹²⁸ bytes = 3.4028237e+38) or usize type which the compiler gets it dynamically from the hardware. So that’s why I implemented BigUint. This library saves in a bytes Vector and allocates memory dynamically.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c75abba90deae854e8ffd41f46cee1f3/href">https://medium.com/media/c75abba90deae854e8ffd41f46cee1f3/href</a></iframe><p>Let’s talk about performance.</p><p>Here, is where Rust domain the panorama.</p><p>First, before executing our Rust script, we must compile it for our machine to understand what instructions need to execute.</p><p>There are 2 ways for that, with rustc &lt;script.rs&gt;<strong> </strong>or with cargo, a tool that manages packages, compile, format, and more. We can do it with cargo build [ — release].<strong> </strong>If the release flag is added, it will optimize the script and it going to execute the code faster than otherwise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EzUaK9z58JAEQyjTDDJRLA.png" /><figcaption>Calculate the 100th Fibonacci number</figcaption></figure><p>After executing with the 100th Fibonacci number we can notice that:</p><p>Rust was executed in 26 µs and Python in 142 µs.</p><p>Obviously, the performance of these two languages cannot be compared. Remember, we didn’t control the memory, the interpreter did for us. This doesn’t mean that Python core developers designed it badly. It is because the code is read first by the interpreter and it checks errors in execution.</p><p>We can put a simple say as if you have to talk to anyone. If both of you understand the language, you will communicate fast. On the other hand, if you need an interpreter to communicate with others, there are more delays in the flow.</p><h3><strong>My Experience</strong></h3><p>I can say Rust is cool, after a big part of the time trying to get the easy Fibonacci solution, I can say the change from Python could be a little hard If you are not used to typing variables, managing memory, pointers, and also all the libraries aren’t standard so you have to look for, try it and continue with the development.</p><p>On the other hand, although it can be a little difficult to start it, Rust has a great <a href="https://doc.rust-lang.org/book/">web book</a> where that introduce you to the language with practical examples with a detailed explanation.</p><p>With a Fibonacci problem, I have been able to verify precisely that the hardware resources we use are not infinite and that python abstracts us from many problems. I let you some resources that could be interesting.</p><ul><li><a href="https://www.rust-lang.org/">Official page</a></li><li><a href="https://doc.rust-lang.org/cargo/">Cargo book</a></li><li><a href="https://towardsdatascience.com/you-want-to-learn-rust-but-you-dont-know-where-to-start-fc826402d5ba">Medium with a lot of resources</a></li><li><a href="https://www.techempower.com/benchmarks/">API Benchmarks</a></li></ul><p>In conclusion, if you are a curious person that likes to learn something new there is a topic with a lot of stuff to look for and where you can join or if you were waiting for any signal to start something new this is your post.</p><p>Thanks for reading, I hope you like it! I’d love to hear your opinion about your language code knowledge or your new learnings.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ee46ee63033" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/from-pythonic-to-rustacean-this-is-the-way-6ee46ee63033">What Happens If We Code the Same Algorithm in Python and Rust?</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Testing, our new hero]]></title>
            <link>https://medium.com/@bergraan/testing-our-new-hero-d238c8388073?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/d238c8388073</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[pytest]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Fri, 03 Dec 2021 17:00:47 GMT</pubDate>
            <atom:updated>2022-03-19T13:20:49.018Z</atom:updated>
            <content:encoded><![CDATA[<p>There is a good lie among developers and that is “In my local it works” but when you deploy someone calls you because some feature is broken.</p><p>if you feel identified, you need to start to test your code and I’m going to show you <strong>why</strong> and <strong>how</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R8vvyFjnUVqkr6Yh.jpg" /><figcaption>The route</figcaption></figure><p>In my experience as a developer (4 years), I learned a lot about frameworks, debugging, languages, etc. I made my apps to practice and apply all my knowledge between my apps and the work projects where I developed. I always had a bug, maybe I could detect it by doing some manual testing before, but when the app grows a lot it is impossible to test all the pieces because you have to spend more time per deployment to check that nothing of your new code is broken.</p><blockquote>It’s not a bug, it’s a feature</blockquote><p>This is not to say that I didn’t do some automatic testing, I did some but it wasn’t the best it could be.</p><p>Since I joined my new job I was taught how I should test code and get the 100% coverage legend. In my previous post about git cli vs ui I mentioned about CI/CD is a way to improve your git skills but it’s more than that. Now, I can deploy on Friday and no one raises a red flag 🚨 They just review my code, and the repository provider runs the pipelines where the tests are launched and auto deploys it.</p><p>Testing will not prevent your code from not having errors, but in case they appear, you can test that case in your code for versions. Let’s see an example.</p><p>Imagine that we have to implement a function to calculate a common bit of the binary matrix.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fe63f0d1a11ef1b6a0273c420016d81d/href">https://medium.com/media/fe63f0d1a11ef1b6a0273c420016d81d/href</a></iframe><p>It can be seen that 3 cases can occur: 1, 0 or None. We can test it manually as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/623cb0a8e27543af5d9ee7cb78b06d58/href">https://medium.com/media/623cb0a8e27543af5d9ee7cb78b06d58/href</a></iframe><p>But sometimes there are moments when someone is hurrying you because client ‘Tier 0’ discovered a bug and they can’t work until you fix it. So, You have to fix it and deploy it as fast without testing it more than the piece of code where the bug was encountered. How we can prevent this? TESTING (You can hear now <a href="https://medium.com/youtube.com/watch?v=Vhh_GeBPOhs">Steve Ballmer</a> shouting “Developers” but with “Testing”)</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fgtm6yZur9eRFoo1UvO%2Fvideo&amp;display_name=GIPHY&amp;url=https%3A%2F%2Fgiphy.com%2Fclips%2Ferichamlet-developers-ballmer-gtm6yZur9eRFoo1UvO&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2Fgtm6yZur9eRFoo1UvO%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="320" height="240" frameborder="0" scrolling="no"><a href="https://medium.com/media/8b2366dbb2fdcf7ce1c0e22a39065c81/href">https://medium.com/media/8b2366dbb2fdcf7ce1c0e22a39065c81/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e24d21f54427ecf3d08e3a4699327987/href">https://medium.com/media/e24d21f54427ecf3d08e3a4699327987/href</a></iframe><p>As you can see, with these 20 lines we test our cases with all the cases we know and we can launch them in a CI/CD to approve that the code works as you have defined before.</p><p>Last but not least, you may think that there are some cases where you can’t test it because the test needs to connect to another service, for example, an API. This is where you have to think that the importance of that functionality is not the service, it’s your code. And python developers know this, that’s why they create some tools to simulate it.</p><p>The answer to the above question is <a href="https://docs.python.org/3/library/unittest.mock.html"><strong>mocks</strong></a>. Mocks are objects that can simulate a <strong>return_value</strong> or <strong>side_effect</strong>. But how? Let’s implement a simple function that calls a PokeApi and depending on the response, returns the name of the Pokemon.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2925e74f3dc99dd51f67384ed7246368/href">https://medium.com/media/2925e74f3dc99dd51f67384ed7246368/href</a></iframe><p>We can test it in the following way:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/61681b3ea6478c74a7ec7b60792f88a3/href">https://medium.com/media/61681b3ea6478c74a7ec7b60792f88a3/href</a></iframe><blockquote>Not every hero wears a cape</blockquote><p>With these tests, we covered 100% of the code, and did not make a request to check different responses to 200 <strong>status_code</strong> as we defined to return None. Explaining the code a bit, we can see that with the <strong>path</strong> decorator we injected the Mock responses class and set attributes as our code checks with whether to return name or None.</p><p>In <strong>conclusion</strong>, we should always test all our components because it can save us from an error in case we edit a piece of code that we think is failing. It is easier to verify our test cases than to read code and think “in my locale it works”.</p><p><strong>Thanks for reading, </strong>I<strong> </strong>hope you like it<strong>! </strong>I would love to hear your opinions or experience with testing. Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>. 😃</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d238c8388073" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Let’s build a theme park with Celery!]]></title>
            <link>https://medium.com/@bergraan/lets-build-a-theme-park-with-celery-8d5a9e1aa91a?source=rss-2fa9776482ea------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d5a9e1aa91a</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[celery]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[learning]]></category>
            <dc:creator><![CDATA[Ángel Berhó]]></dc:creator>
            <pubDate>Sun, 14 Nov 2021 15:02:29 GMT</pubDate>
            <atom:updated>2021-11-14T15:02:29.520Z</atom:updated>
            <content:encoded><![CDATA[<p>Hi everyone, as you see in the title of the Post we will make a Virtual theme park🎢 with Celery 🙌. I’m inspired because I was on vacation there and I noticed some concepts are similar when we work with async tasks.</p><blockquote>I will include my own disclaimer. Maybe some of concepts of the Theme parks are not realistic but similar. This is a didactic post that lets you learn concepts about the Celery async task framework.</blockquote><h3>What is Celery?</h3><p>As his documentation says: <em>Celery is a task queue with batteries included. It’s easy to use so that you can get started without learning the full complexities of the problem it solves. </em>What does that mean? it allows you to run python asynchronous code when you send an event in another process without any limitation of load. Read more <a href="https://docs.celeryproject.org/en/stable/">here</a></p><p>Let’s start with a simple API example. Imagine you have an API that receives information from a form. API should extract the data, transform, and load. It’s usually called <strong>ETL</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*PHuauDA8tDhDK7acQBSvxQ.png" /></figure><p>API usually should be sync, so the client is waiting for any response and that’s why if the server does not respond in some seconds it will return a <strong>timeout.</strong> So, the API has to validate incoming data and send it as an event to another service that can process this data and be async. Like the next image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*INMTNUY-8v4xNqB2jBT9iw.png" /><figcaption>Celery data flow</figcaption></figure><p>In the image, we can see some processes/actors:</p><ul><li><strong>User</strong>: who initiates the flow.</li><li><strong>API</strong>: Receive data and check incoming data.</li><li><strong>Broker</strong>: Where the event was sent.</li><li><strong>Celery</strong>: Worker/s who execute the tasks.</li></ul><h4>Why this data flow?</h4><p>The user usually doesn&#39;t care when his information is processed, he only matters the final result of the operation and displays it in any beautiful dashboard. So API is one process where responsibility is to get data from external sources and doesn’t block to the user, you know, the loading spin.</p><p>After this short explanation, we are one step closer to making our virtual theme park. Let’s continue!</p><h3>Amusement ride</h3><p>To create a virtual theme park, we need some amusement rides. I made a picture for you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y070Ee_AQej4gRyc8kLQKg.png" /><figcaption>Grand canyon rapid — Port Aventura</figcaption></figure><p>We must conceptualize this as:</p><h4>Worker</h4><p>In this case, the workers are the boats. Each boat has 8 seats, that means can process 8 users/tasks per worker, also called <strong>concurrency</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*4Zm2lA2jNSEc_coMTC5yBw.png" /></figure><h4>Server</h4><p>The river, if any task overloads the resources of the server it will die before turning down the server. In your logs you will see something like:</p><pre>WorkerLostError: Worker exited prematurely: signal 15 (SIGTERM).</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*quz1PaOBRp4gaKjkHSudkw.png" /></figure><h4>Queue</h4><p>Our channel is the bridge where all the users are waiting to ride it. In the celery workflow, the queue is something virtual, broker saves it until any <strong>Consumer </strong>gets the task. Brokers usually save this information in ram, so you have to take care of this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HE3Mt4db5jFwQGLqx6cr1w.png" /></figure><h4>Tasks</h4><p>Here are very simple to identify, each person is a task that must be executed at any time. They wait for some wet fun in the boat.</p><h4>API</h4><p>Who receives the data from users in this case is the door and staff who validate if the person can ride (Height validator).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*K3smWAwJAasMqappdZcspA.png" /></figure><h4>Execution</h4><p>It’s hard to show it in an image because it should be from the user sitting in the boat, going down the rapids, and sitting up.</p><p>The final process is like the next image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*I1gE0i-PvtOAoQgqQI_8sA.png" /></figure><p>Easy! we disengage all pieces and we can develop our Amusement ride as code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/84ff56e45c31db72045dbfb83372f6d3/href">https://medium.com/media/84ff56e45c31db72045dbfb83372f6d3/href</a></iframe><p>Now, when we launch dispatch_grand_canyon ‘s task, the process will instance the user journey and add the actions on his own.</p><p>A broker can’t be developed because is part of the infrastructure, in this case, we can use RabbitMQ or Redis who manage events as Pub-Sub pattern from a queue.</p><h4>Queue</h4><p>The queue is <strong>part of the infrastructure</strong>, You should think about it as the physical part of the process just like the worker is a server with <strong>CPU</strong> and <strong>RAM</strong>. We said before, Celery uses <strong>RabbitMQ</strong> or <strong>Redis</strong> as Broker which is the channel to send events between any service and celery.</p><p>The queue is a stack of tasks, in the Grand canyon example, users were the tasks, and the bridge was the queue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*OxAYNKEeTKMVWBd-AfCDhQ.png" /></figure><p>Remember, theme parks usually have 2 queues, the <strong>normal</strong> queue, and the <strong>express</strong> queue where you can avoid the other and execute tasks as a high priority. How do they conceptualize this?</p><p>When I was there I saw they divide into 2 per boat the number of tasks/users. A similar way of doing this is to create another task and configure it to execute this task in another queue.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51d3df09a7c1e6ba5283616b419ec77c/href">https://medium.com/media/51d3df09a7c1e6ba5283616b419ec77c/href</a></iframe><p>We defined per task a queue in the celery configuration. Now in the infrastructure, we need 2 workers:</p><ol><li>For <strong>normal</strong> queue: celery -a &lt;project_name&gt; -c=3 -Q grand_canyon_normal options set in the command at run process are 3 of concurrency and queue <strong>grand_canyon_normal</strong>.</li><li>For <strong>express</strong> queue: celery -a &lt;project_name&gt; -c=3 -Q grand_canyon_express options set in the command at run process are 3 of concurrency and queue <strong>grand_canyon_express</strong>.</li></ol><p>It’s equal to having 2 workers working in parallel running the same code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*QETXvPdSgWVAeMnb9W2HOg.png" /></figure><p>The way of Theme parks scale-up Amusement rides is by adding more vehicles. For example, in the grand canyon, they have like 5 or more boats, in Furious Baco 2, and so on. Another way to maintain a queue is by adding more distance, this is equivalent to buying more Ram on the server because you can save more events without overflowing the memory while the workers are processing the work but it can be frustrating to wait while you see others be funny in the boat 😜.</p><p>I hope I could help you to learn how Celery works. This is the tip of the iceberg of what Celery can do. We will see in-depth in another post!</p><p><strong>Thanks for reading! </strong>I’d love to hear your opinion. Leave a comment or contact me on <a href="https://www.linkedin.com/in/%C3%A1ngel-berh%C3%B3-grande/">LinkedIn</a>. 😃</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d5a9e1aa91a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>