<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Victorfjansen on Medium]]></title>
        <description><![CDATA[Stories by Victorfjansen on Medium]]></description>
        <link>https://medium.com/@victorfjansen?source=rss-6d568ceabee5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*srHn16JVxY_2kangFnyI4g.jpeg</url>
            <title>Stories by Victorfjansen on Medium</title>
            <link>https://medium.com/@victorfjansen?source=rss-6d568ceabee5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:08:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@victorfjansen/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Feistel Network — SPACE Cryptography Analysis]]></title>
            <link>https://medium.com/@victorfjansen/feistel-network-space-cryptography-analysis-75cabc195d58?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/75cabc195d58</guid>
            <category><![CDATA[cryptography]]></category>
            <category><![CDATA[feistel-cipher]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Sun, 30 Aug 2026 02:53:53 GMT</pubDate>
            <atom:updated>2026-08-30T02:53:53.090Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the first things I’ve learned when studying criptography was Feistel Network. Somehow it opened my mind to think in criptography in a different way.</p><p>I even didn’t know about “cryptography frameworks” existence and how to use it. So, maybe you don’t know as well.</p><p>Not only feistel network’s interesting, but also other ciphers built with it. An example I’ll bring here is about <strong>SPACE cipher family. </strong>Just read it and have fun!</p><h4>Feistel Network</h4><p>The Feistel Cipher is a design model used to build block ciphers, such as SPACE. It provides a model to create secure encryption algorithms.</p><p>This method basically follows 3 steps and they are:</p><ul><li>Convert Plaintext to Binary</li><li>Divide in 2 parts</li><li>Generate round keys</li></ul><p>Simple, no? So let’s see an example of 2-round feistel cipher.</p><ol><li>After converting plaintext to ASCII and then to binary, we’ll divide it in two halves. We will call the first half R1 (right 1) and the second one L1 (left 1)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*batpMebyG3lt0-WSJki-TA.jpeg" /></figure><p>2. We will create a K1 and K2 — random binary keys. Each key should have a length equal the halves.</p><p>Then, we start the rounds:</p><ol><li>The first round:</li></ol><p>Compute Function 1 — F1:</p><ul><li>F1 = R1 XOR K1</li></ul><p>Update Halves:</p><ul><li>R2 = L1 XOR F1;</li><li>L2 = R1</li></ul><p>2. The second round:</p><p>Compute Funcion 2 — F2:</p><ul><li>F2 = R2 XOR K2</li></ul><p>Update Halves:</p><ul><li>R3 = L3 XOR F2</li><li>L3 = R2;</li></ul><p>And then, maybe you’ve noticed we’re placing in each round the Ln value with the Rn-1 value. We can see it through a visual representation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*ck_sEPsFpnm71dZZEXPnaA.png" /></figure><p>To be more tangible, take a look on this python code snippet:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f0f7560bcc348d35e51cc383779d6935/href">https://medium.com/media/f0f7560bcc348d35e51cc383779d6935/href</a></iframe><h4>SPACE Cryptography Family</h4><p>The SPACE cryptography was introduced on <em>“White-box Cryptography Revisited: Space-Hard Ciphers”</em> paper. You can check it here:</p><iframe src="https://drive.google.com/viewerng/viewer?url=https%3A//static.aminer.org/pdf/20170130/pdfs/ccs/05mbdq4mgwku6ewzpclfldt7hfbek1an.pdf&amp;embedded=true" width="600" height="780" frameborder="0" scrolling="no"><a href="https://medium.com/media/a875f70072c134a78d979db22ff42b6e/href">https://medium.com/media/a875f70072c134a78d979db22ff42b6e/href</a></iframe><p>To put it simply, <strong>how does it work</strong>?</p><p>SPACE isn’t trying to reivent the wheel from scratch. It takes a trusted block cipher (AES-128) and wraps it on a<strong> Unbalanced Feistel Network</strong>;</p><p>The main difference is: now the halves are uneven. Out of the full 128-bit block, SPACE only gets a small chunk of Nin bits (the designers recommend instantiations equal to 8,16,24,32), and pass through the round function, leaving the remaining bits to be XORed with the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*o2yYNkJy73hbHJ6BQeL1SQ.png" /></figure><p>So, as we looked step by step on rounds implementation — Pure Feistel Network — this is how SPACE round works:</p><ol><li>Slice: Extract the first Nin bits from 128-bit block</li><li>Transform (Table Lookup): Pass those Nin bits into a lookup table T. This table outputs a larger mask of 128-Nin bits.</li><li>XOR: Combine this output with the rest of the 128-bit block using an XOR operation.</li><li>Rotate: Place the original Nin bits at the end of the block, shofting everything down so a new slice is ready for the next round</li><li>Repeat. As many as you want.</li></ol><p>So, again, to not be only in theory, here’s a python code snippet to be more clear:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d11e10e08acda8f528e743012ea41d26/href">https://medium.com/media/d11e10e08acda8f528e743012ea41d26/href</a></iframe><p>And what’s the advantage of using it — considering the white-box approach?</p><p>Maybe you’ve noticed, but our table stores the pre-computed outputs for all possible Nin-bit inputs mixed with our secret key.</p><p>This lookup table consumes way more memory, but because the key is directly baked into these pre-calculated values, the algorithm never loads the key into memory — making it extremely hard for an attacker to extract it.</p><p>That’s all. Hope you enjoy it, and I’ll be bringing a way to measure the hardware impact when using SPACE cryptography.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75cabc195d58" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How YOU could run AI models in the browser using Javascript]]></title>
            <link>https://medium.com/@victorfjansen/how-you-could-run-ai-models-in-the-browser-using-javascript-c961e772b91b?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/c961e772b91b</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Tue, 25 Feb 2025 08:44:33 GMT</pubDate>
            <atom:updated>2025-02-25T08:44:33.317Z</atom:updated>
            <content:encoded><![CDATA[<p>The idea here is to share a way of running AI models without needing backend servers, with just your browser and device.</p><p>You can run it in different ways with different resources, for example. CPU, Web GPU, and so on. However, in this article, we’re going to run with WASM.</p><p>WASM (web assembly) is a binary format that allows us to execute code in web browsers. It operates as a low-level bytecode in an isolated environment within the browser, which leads to better performance in web environments.</p><p>The choice was made based on the type of model we want to run — light, non-complex models — and because I want it to run on more compact devices such as smartphones and so on…. (hardware will have an impact).</p><p>To be clear, I’ll be using Angular as the frontend framework because of the RxJS/observer pattern. The architecture makes it quick and easy to use, and it’s optimized for this purpose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZNEGqIgTr3VyzE9ZS_AgPQ.png" /></figure><h4><strong>A friendly advice</strong></h4><p>I recommend that you know about Buffers, Streams, and Workers because they are topics that I will cover, but not necessarily explain.</p><p>You need to know these <strong>basic</strong> concepts to understand what I’m doing here properly.</p><p>You can learn more by reading other articles I’ve written here on the platform.</p><ul><li><a href="https://victorfjansen.com/writable-streams-in-nodejs-hardware-impact-analysis-bd9002e82a60">Writable Streams in NodeJs— Hardware Impact Analysis</a></li><li><a href="https://victorfjansen.com/encryption-decryption-stream-on-demand-nodejs-approach-90a7e5e3bdd2">Encryption/Decryption Stream on Demand — NodeJS approach</a></li></ul><h4>First of all, why run AI models in the browser?</h4><p>With the idea of small models running in the browser, we can have some benefits when it comes to saving computing resources on the server.</p><p>Let’s think about the following scenario:</p><ul><li>I have a car sales system.</li><li>To sell a car, I need to send an external photo for the analysts to evaluate and give a price.</li></ul><p>In this scenario, we have one main concern:</p><ul><li>Is the user sending me a photo of a car?</li></ul><p>I can create an asynchronous service where the analyst will evaluate or I can create an asynchronous service where an AI will evaluate whether it’s a car.</p><p>All this comes at a cost. These are all computer resources. It’s all money 💵💰.</p><p>What if I could create an extremely lightweight model to classify whether a photo is of a car or not? What if the model was at least 80% accurate in making this identification?</p><p>This would bar a huge part of the total cost of this validation, as well as the labor of the analysts.</p><p>Voila, we have a real reason to use this type of technology.</p><p>This may be an experimental scenario, but in the long run, it will certainly be an incredible utility that will help many people and companies.</p><h4>And how do you decide which model to use?</h4><p>Well, that depends on your needs. It’s a bit complicated how to evaluate, but you have to keep a few things in mind:</p><ul><li>Even though today’s devices are increasingly arriving with absurd data processing capacity, don’t always consider the best-case scenario.</li><li><strong>The lighter the model, the better.</strong></li><li>Be sure of your model’s metrics (such as accuracy, loss, and f1 score) and, if possible, create your models.</li></ul><p>To get the best metrics, it’s important to have contact with an ML engineer or at least understand the fundamentals behind neural networks.</p><p>An extremely important topic for this type of decision is the architecture of a neural network. It shows how many layers there are and this directly influences the size of the model generated.</p><p>In addition, there are issues such as processing between the layers of the neural network. We have, for example, the <strong>VGG</strong> architecture which is widely used in convolutional networks. <strong>VGG</strong> is an architecture known for being heavier and a little older.</p><p>Another example — and one that I would even recommend for this case — is <strong>MobileNet</strong>.</p><p><strong>MobileNet</strong> is a type of neural network designed to perform exceptionally well on mobile devices and embedded systems. In particular,<strong> I believe it is one of the best </strong>— if not the best — type of model to choose.</p><p>I recently published a paper in which we analyzed the influence of a neural network’s architecture on distributed training.</p><p>We used a MobileNetV2_100 model with 289 internal layers and<strong> achieved a classification with 96% accuracy and only 8.8MB of file </strong>— our scenario was the classification of rice diseases.</p><p>This model is excellent for such a purpose. It’s light, simple, compact, and will run on a variety of devices.</p><p>Given this scenario, although it is interesting to use ready-made models, you can <strong>create your models</strong> and structure them according to your needs after studying a little about neural network architecture and looking up some articles on the subject. However, for the time being, I’m not focused on teaching this here — that will be the subject of a future article.</p><blockquote>If you’re interested in finding out more:</blockquote><p><a href="https://doi.org/10.22456/2175-2745.143508">Data Augmentation and Convolutional Network Architecture Influence on Distributed Learning</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fplaylist%3Flist%3DPLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi%26si%3Dof3eyXCr9XeYi40V&amp;image=https%3A%2F%2Fi9.ytimg.com%2Fs_p%2FPLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi%2Flandscape_mqdefault.jpg%3Fsqp%3DCJD39L0Gir7X7AMICNGyiNsFEAE%3D%26rs%3DAOn4CLDDoK4Vbce33FjYrEthmu8xm8OVAw%26v%3D1533155665%26days_since_epoch%3D20144&amp;type=text%2Fhtml&amp;schema=youtube" width="853" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b2dfbaa618681d7b2163333980b5abcd/href">https://medium.com/media/b2dfbaa618681d7b2163333980b5abcd/href</a></iframe><p>Perfect. Now to our project.</p><h4>Where we want to go</h4><p>What we’re going to do is run an audio transcription model in real-time, using the Transformers.js library from hugging face (which runs based on the Onnx model — we’ll talk about it in a moment), but implementing some additional features. These are:</p><ul><li><strong>Web Workers: </strong>Ensuring that our UI doesn’t crash due to processing or bring a loss of performance to our device.</li><li><strong>Async Generators e Streams</strong>: Split large audio files into chunks and process audio incrementally for real-time transcription feedback.</li><li><strong>Web Assembly:</strong> use ONNX runtime via Transformers.js to run the Whisper-tiny model at near-native speed.</li></ul><p>These features are native to Node.js and we’ll take advantage of them without worry. If you don’t know some of them,<strong> I recommend looking them up before proceeding.</strong></p><p>In addition, I’ve deployed the project and you can check it out at:</p><p><a href="https://ai.victorfjansen.com/">Transformers</a></p><p>The repository is also available with all the code ready, so if you don’t want to read it, feel free to access it:</p><p><a href="https://github.com/victorfjansen/AI-Speech-Conversion-Runner">GitHub - victorfjansen/AI-Speech-Conversion-Runner</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*XZ3sV794hjnnWJ_G2N2EoA.gif" /><figcaption>ai.victorfjansen.com running Ai in Browser</figcaption></figure><h4>Whisper — Our beloved model</h4><p>OWhisper is the guy for all kinds of text recognition work. It supports several languages and was created for speech recognition, language translation, and language identification.</p><p>It is a more than complete model created by OpenAI and there are several versions of it with different weights. Naturally, heavier models will be more powerful but will require more hardware to process.</p><p>Here, we will use 3 main versions:</p><ul><li>Tiny — lighter version, consisting of 39M parameters</li><li>Base — version above the light version, but with 74M parameters</li><li>Turbo — the latest version released, but with 844M parameters</li></ul><p>A rather curious leap in parameters, isn’t it?</p><p>The idea here is to stress the scenery a little, so lighter models (which will work better) should be tested and a larger model taken into consideration.</p><p>However, feel free to try out other models! I’d be flattered if you had any comments about your experience with different versions of what I’m going to show you here.</p><p>The official repository with the current architecture can be found on github:</p><p><a href="https://github.com/openai/whisper">GitHub - openai/whisper: Robust Speech Recognition via Large-Scale Weak Supervision</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cMke9qdC4EvowVEDquVCjw.png" /><figcaption>Whisper Approach</figcaption></figure><p>Our first problem: <strong>The conventional model doesn’t work for us.</strong></p><p>We could, for example, download directly from the official repository or use some package to obtain these models from a private artifactory, but we need a model adapted to run with <strong>ONNX</strong>.</p><p>And then comes the question:</p><h4>What the hell is ONNX?</h4><p>Onnx is a format for the interoperability of models trained in different frameworks.</p><p>If, for example, I train one model using Pytorch and another using Tensorflow, I can have a standardized directory for both models. Just convert to .onnx and use.</p><p>The aim is to be able to versatilize in different environments, especially when it comes to embedded devices and browsers.</p><p>In addition, Onnx has its runtime. So, in addition to standardizing the files, it can run the standardized model using CPU, GPU, Wasm and so on.</p><p>That’s why we need to use an Onnx model. Interoperability of models and ability to run in the browser.</p><p>Thank God we have a huge community on HuggingFace from Onnx and all the models we need are already available without the need to convert!</p><p><strong>But if you have your own trained model,</strong> you can make this conversion easily. Just look up how to do it and I’m sure that in no more than 2 or 3 lines of code, you can convert your model.</p><p>To access the community, you can click here:</p><p><a href="https://huggingface.co/onnx-community">onnx-community (ONNX Community)</a></p><p>Now that we know what Onnx is, what we’re going to use, why we’re going to use it and the benefits of using it, we have to talk about our architecture!</p><h4>A bigger vision — Our architecture and implementation</h4><p>Right, moving on to the project requirements, we need to think very carefully about how this real-time communication and transcription is going to take place.</p><blockquote><strong>A warning: I’m going to go through function by function, explaining how the processing and theory will work in each part. I recommend that you try to understand it at first. The final code with everything integrated will be at the end of this session!</strong></blockquote><p><strong>Audio Context — streaming audio chunks</strong></p><p>We have the following requirements:</p><ul><li>Let’s send an audio.</li><li>The audio has to be “understandable” to the model</li><li>We have to process this audio in parts so as not to overload memory or the CPU</li><li>After processing, we must return to the user so that they can see the result.</li></ul><p>Okay, we have no worries about the <strong>first requirement</strong>. If you’ve worked with the front end at all, you’ll know that the idea is to create a “file” input and send the desired file. In this case, all we have to do is limit the file sent to the “audio/*” type.</p><p>For the<strong> second requirement</strong>, we already have some concerns. After all, what would it mean to make the audio “understandable” to the model?</p><p>Every file, every interface, everything that exists in the world of computing is a buffer. Or rather, “little boxes that can store the file in the hexadecimal format”. In the end, everything is converted to binary so that we can handle it the way we want.</p><p>An audio file is no different. It’s made up of binary, all we have to do is interpret it.</p><p>As we are using Javascript, we have a great companion for this moment called <strong>“AudioContext”.</strong> <strong>AudioContext</strong> is a web api that allows us to transform audio into Buffers. This way, we’ll use all the content provided by this API to interpret our audio.</p><p>And how do we do that? See below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d75569124df95a6ef44c05a6606bba31/href">https://medium.com/media/d75569124df95a6ef44c05a6606bba31/href</a></iframe><p>Please note:</p><ul><li>We created an AudioContext with a sampleRate of 16000.</li><li>We create the file buffer with file.arrayBuffer();</li><li>We create the audio buffer, decoding it with audioContext.</li></ul><p>It’s as simple as that. And you may still be asking me<strong> what the sample rate is</strong>. Quickly:</p><ul><li>The sample rate is an audio sampling rate, measured in audio samples captured per second. In this case, it is measured in Hertz (Hz). So we’re setting our audio to 16_000 Hz.</li><li>And why 16_000? Because the Whisper model was trained with 16_000 Hz. If we don’t set it as the default, we’ll have some problems when translating or transcribing audio — I invite you to do some tests and see for yourself!</li></ul><p>We also have another issue: <strong><em>the audio channels.</em></strong></p><p>After going through this transformation with the AudioContext API, you have audio channels available to create a sense of space and depth.</p><p>You know when you go to buy a phone and the specs say it has stereo sound? Well, that’s exactly what it is. It’s two channels that give you a sense of space and depth.</p><p>However, each channel has its buffer and this can be a problem. We can’t handle separate elements in audio identification. Therefore, we have to convert to mono (only 1 channel).</p><p>Thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9fcf6541ad1df75f9729eaa568c41fb0/href">https://medium.com/media/9fcf6541ad1df75f9729eaa568c41fb0/href</a></iframe><p>Please note:</p><ul><li>We have the same code as before for creating the audio buffer.</li><li>If there is only one channel in the audio buffer, we return the one with index 0.</li><li>If not, we average each value and return a buffer of equal size for all indexes!</li></ul><p>This way, we create a mono channel for our dual-channel audio.</p><p>Overall, we will have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ofu0ZedmpzYtESE4OLK5qA.png" /><figcaption>flow</figcaption></figure><p>For the <strong>third point</strong>, we have audio processing in parts so as not to overload the front end.</p><p>Honestly, that’s the simplest part, I’d say. If you’ve read the articles I’ve written before — which I’ve left at the beginning of this one — you already know how to do it.</p><p>We have to create a “disposer” and calculate the already-used audio offsets. This turns into the classic iterator with i++, where we add a value to the value already traveled.</p><p>To do this, we’re going to use the native JavaScript feature called <strong>generators</strong>. Generators are functions that provide data dynamically without storing or processing all the information unless requested.</p><p>A simple example for you to understand:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0b45d85d6583b43ab8c5ed10efc060ca/href">https://medium.com/media/0b45d85d6583b43ab8c5ed10efc060ca/href</a></iframe><p>Please note:</p><ul><li>The counter function “yields” one value at a time.</li><li>We don’t have a wealth of information. Once it’s been passed on, we progress to the next result.</li></ul><p>The idea is the same. Take the values according to dynamic generation and use them to count the chunks.</p><p>That way, we’ll have something like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/538fef06561579ef118390bc9691e4a3/href">https://medium.com/media/538fef06561579ef118390bc9691e4a3/href</a></iframe><p>Note:</p><ul><li>It’s the same code I presented earlier, converting the Audio Buffer.</li><li>We are creating an offset, iterating over it, and delivering the existing content inside the chunk to the caller with async generators.</li><li>We have a parameter called “chunkSizeSeconds”. It will determine the size of the chunk. Note that the constant “samplesPerChunk” uses this size in seconds to calculate the default size of the processed buffer.</li><li>Remember that the sampleRate is an audio sample. If we have 16_000 samples per second and we’re considering chunks of 30 seconds, we have to do this multiplication to get the total size of the number of seconds we want.</li><li>At the end, we close AudioContext so that it doesn’t consume memory.</li></ul><p>To close with a bang, we’ll implement the buffer processing function from the async generator.</p><p>Thus, we have:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2032d73ba56579a316168b02bd5eb1ef/href">https://medium.com/media/2032d73ba56579a316168b02bd5eb1ef/href</a></iframe><p>Note:</p><ul><li>We always create a new audioContext for processing a batch of Buffers</li><li>We PostMessage the Chunk to the current worker. Thus, we process chunk by chunk coming from the async generator.</li></ul><p>In this implementation, if we don’t handle it, we’ll have some problems with asynchronism and sequential information. Later we’ll talk about asynchronous queues and how we solve each aspect of processing.</p><p>Now that we’ve talked about the whole audio context and the transformations needed to run the model, we can move on to understanding the pipeline and workers.</p><p><strong>Model Pipeline — Optimizing with Worker Threads</strong></p><p>Our audio service has all the functions we mentioned above. Now we need to process the chunks generated earlier in our worker.</p><p>To begin with, let’s talk about the Transformers.js lib. It alone can take these chunks and process them within the Onnx runtime based on the model we have defined.</p><p>We can even use online models, without the need to store local models, but it will require the client’s bandwidth to load the model.</p><p>Its implementation is very simple. We don’t need much to run it. Thus, we have:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72224ebf593ff48e335c53b13123a33d/href">https://medium.com/media/72224ebf593ff48e335c53b13123a33d/href</a></iframe><p>Note:</p><ul><li>The pipeline is created by passing the family of the model you want to use.</li><li>The result is given by invoking the pipeline created as a function.</li><li>There are many different types of pipelines. You can explore and learn other methods of data transfer, in addition to audio processing.</li></ul><p>You can still define the model you want — check availability in the official hugging face repository. In this way, we have</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0bb07e119a184f9482d2239651d9ff4e/href">https://medium.com/media/0bb07e119a184f9482d2239651d9ff4e/href</a></iframe><p>Note:</p><ul><li>I’m still using the model for sentiment analysis (first parameter)</li><li>I’m defining the model I want to implement (second argument)</li><li>I also added the type of processing I’d like to implement. In this case, it’s webgpu.</li></ul><p>Now, for our scenario, we want to place our model (I’ll use local). To do this, I need to use a model compatible with Onnx and clone its repository to my machine.</p><p>Remember the hugging face link I mentioned earlier? Then we’ll use it.</p><p><a href="https://huggingface.co/onnx-community">onnx-community (ONNX Community)</a></p><p>This way, you can search the community for an available template or just convert the template itself and store it in the public/models folder — as I mentioned earlier, I’m using Angular as my frontend framework.</p><p>Thus, accessing the hugging face:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P8KE-nMsb1NnB1idAT3xUQ.png" /></figure><p>Accessing the model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qy3Umu4i8v0knFmV18x-VQ.png" /></figure><p>And cloning the repository:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_-uXu7dsKbAXilQJrnsppA.png" /></figure><p>We’ll have our models in hand:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/774/1*dSOQCA-UaYc3HLZt8i_Zkw.png" /><figcaption>Each folder is a different model</figcaption></figure><p>Now that we have our models, we need to follow a few steps:</p><ul><li>Create a worker</li><li>Load the models into the worker</li><li>Process incoming chunks</li></ul><p>For the first step, the most recommended way is to follow the Angular CLI. This way we create a worker from scratch and configure tsconfig.worker.json automatically.</p><p>Thus:</p><pre>ng generate web-worker &lt;location&gt;</pre><p>We will then have a file like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8fb72259dfeca83acf3e06084eef000e/href">https://medium.com/media/8fb72259dfeca83acf3e06084eef000e/href</a></iframe><p>Note that a worker is an EventEmitter. We have a few event listeners that we will use, the main one being “message”. We’ll use the “message” to communicate I/O with the worker.</p><p>Okay, now we can download the Transformers.js lib:</p><pre>npm i @huggingface/transformers</pre><p>And, within our worker, define some properties about where the worker will look for the models to load. These settings refer to the huggingface lib. We need them to be able to load our models correctly.</p><p>Thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/95f163768fbddaa16f431f4d19469e11/href">https://medium.com/media/95f163768fbddaa16f431f4d19469e11/href</a></iframe><p>See:</p><ul><li>We enable local models</li><li>We define the root folder where the models will be allocated (as we defined earlier: public/models)</li><li>We disable remote models. In other words, it won’t consider any external models.</li></ul><p>The next step is to create the pipeline itself and add it to a global worker variable. That way, as long as that instance of the worker is alive, it will load the same pipeline for all operations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97f330d79bcf4f18c8af37daeba26780/href">https://medium.com/media/97f330d79bcf4f18c8af37daeba26780/href</a></iframe><p>Note:</p><ul><li>We set the model type to “automatic-speech-recognition” for text recognition. There are several other types you can explore.</li><li>The model name will appear in the worker data, but if it doesn’t, try opening it with “whisper-tiny/”. The “whisper-tiny/” is the path of the model found in public/models. So if you have other models, you can direct the pipeline to open just by entering the name of the current directory.</li><li>To ensure that the model looks at the internal files, we set local_files_only to true.</li><li>And the device as “wasm” for loading as WebAssembly.</li></ul><p>Once we have our instances, we can add the main eventListener processing our chunks.</p><p>Thus:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/300cbb47af7a45608982f1fc740a4542/href">https://medium.com/media/300cbb47af7a45608982f1fc740a4542/href</a></iframe><p>See:</p><ul><li>Our worker has an asynchronous callback to handle our queue.</li><li>Every time there is a new message, we add the processingQueue chained with a .then. This way, the chunks that will come sequentially will have to respect each other’s order and only start the process if the previous one is finished.</li><li>If the pipeline instance does not exist, it must be mounted.</li></ul><p>And the most interesting is in the following block:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5ee1d496bdaca264671b848d7d6564f6/href">https://medium.com/media/5ee1d496bdaca264671b848d7d6564f6/href</a></iframe><p>We’re dealing with a stream! Where the TextStream is responsible for taking the data from the tokenizer and processing it each time the Buffer is filled.</p><p>The callback_function takes “result” as its parameter, which in turn is the string currently being processed!</p><p>WhisperPipeline defines the current stream and some other properties that are more than obvious from the context.</p><p>In this way, we set up our worker with parallel processing, returning information in real-time without crashing the main thread.</p><h4>Integrating everything:</h4><p>I tried hard to make the concept clear, but the implementation of it — if you tried to follow it — must have been messy.</p><p>I’ll leave the code pointing to the repository where you can consult each of the 3 files and their actual implementation. How are the workers, streams and so on being created?</p><p>Transformers Service — Audio Processing:</p><p><a href="https://github.com/victorfjansen/AI-Speech-Conversion-Runner/blob/main/src/app/services/transformers/transformers.service.ts">AI-Speech-Conversion-Runner/src/app/services/transformers/transformers.service.ts at main · victorfjansen/AI-Speech-Conversion-Runner</a></p><p>Whisper Worker — Model Pipeline:</p><p><a href="https://github.com/victorfjansen/AI-Speech-Conversion-Runner/blob/main/src/app/workers/whisper/whisper.worker.ts">AI-Speech-Conversion-Runner/src/app/workers/whisper/whisper.worker.ts at main · victorfjansen/AI-Speech-Conversion-Runner</a></p><p>DI Token:</p><p><a href="https://github.com/victorfjansen/AI-Speech-Conversion-Runner/blob/main/src/app/tokens/whisper/whisper-worker.token.ts">AI-Speech-Conversion-Runner/src/app/tokens/whisper/whisper-worker.token.ts at main · victorfjansen/AI-Speech-Conversion-Runner</a></p><h4>Additional settings</h4><p>In addition to the core of our application (explained above), we have the front end itself. Layout, design, buttons, and so on.</p><p>My focus here is not to go through this, so I won’t go into these basic implementations.</p><p>If you’re interested, just check out the App.component.html and App.component.ts files. The triggers will be configured there.</p><p>However, there is a setting to be made in angular.json. Only with it was I able to run in development.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/47b271f512649c68dd1d2c6336e6d549/href">https://medium.com/media/47b271f512649c68dd1d2c6336e6d549/href</a></iframe><h4>Conclusion</h4><p>Here we go through topics such as: Creating webworkers and asynchronous queues, Using Streams for dynamic processing of generated words, Audio conversion from practice to theory, WebAssembly and loading modules with the pipeline object.</p><p>I know there are other concepts involved, but I strongly believe that the references I’ve left with the comments will be useful. So, a request from the author is: to read and re-read. Do your research and if I haven’t been clear on any concept, let me know here and I’ll explain and correct the text without any problems.</p><p>Thank you for reading this far.</p><p>Project repository with everything involved:</p><p><a href="https://github.com/victorfjansen/AI-Speech-Conversion-Runner">GitHub - victorfjansen/AI-Speech-Conversion-Runner</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c961e772b91b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Encryption/Decryption Stream on Demand — NodeJS approach]]></title>
            <link>https://medium.com/@victorfjansen/encryption-decryption-stream-on-demand-nodejs-approach-90a7e5e3bdd2?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/90a7e5e3bdd2</guid>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[protocol-buffers]]></category>
            <category><![CDATA[encryption]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Thu, 13 Feb 2025 04:24:49 GMT</pubDate>
            <atom:updated>2025-02-13T12:26:46.858Z</atom:updated>
            <content:encoded><![CDATA[<h3>Encryption/Decryption Stream on Demand — NodeJS approach</h3><p>The right way to encrypt your huge data on demand, with no memory leaks and no hardware impacts!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fl5rJEYZuONBdLHmnpwkJQ.png" /></figure><p>For you that is new here, we are beginning a new series of articles, following the NodeJS approach.</p><p>In my last article, I wrote about NodeJS streams and their impact on machine hardware. I created many and many cases to show how we could handle the same scenario — which was write one million logs in a file — with:</p><ul><li>Callbacks</li><li>Asynchronous code and fs/promises module</li><li>Synchronous code and fs module</li><li>Streams with memory leaks</li><li>Streams without memory leaks</li></ul><p>The article is available and I recommend that you read it before we start here:</p><p><a href="https://victorfjansen.com/writable-streams-in-nodejs-hardware-impact-analysis-bd9002e82a60">Writable Streams in NodeJs— Hardware Impact Analysis</a></p><h4>A friendly advice:</h4><p>In the article that I mentioned here, I wrote about buffers, binaries, hexadecimal numbers, how NodeJs handle these objects in Streams, and also about memory allocation in NodeJS.</p><p>Please, to not be lost here, read the article.</p><h4>A little introduction about Readable, Duplex, and Transform streams</h4><p>If you are familiar with Streams, you should know this representation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*URStOboOcn9xAUV-cWmmjQ.png" /></figure><p>Some obvious annotations:</p><ul><li>Streams have internal Buffer objects to handle data (or chunks).</li><li>Streams are EventEmitters (subject that we already talked about in our previous article), and communicate by observing events as “data”. You can observe this behavior by seeing the last arrow getting out from the buffer representation.</li><li>Streams do not have necessarily 16834 bytes. It can have as many bytes as we want, it just simply changes the “highWaterMark” property.</li></ul><p>So, unlike the Writable Streams, now we are reading from some source by separating the source into chunks. That&#39;s the principle.</p><p>And how we can implement a readable stream to read one million numbers, for example? See:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ad739e42da70347f0bbc923c6d03895e/href">https://medium.com/media/ad739e42da70347f0bbc923c6d03895e/href</a></iframe><p>Please note:</p><ul><li>We’re using writable streams to copy another file with one million rows — each row has 1 number — and I built this file with the code of the last article — it just took 200ms to build.</li><li>The read stream uses the event “data” to get each chunk — one for time — and write it at our write stream.</li><li>The read stream can be paused and resumed just by using simple commands. The idea is that when buffers get filled, the write stream stops and just resumes at the “drain” event — which is the event charged with emptying the buffer.</li></ul><p>This process took just 14ms to complete:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9psuDjmVtywVCUncg0OSNQ.png" /></figure><p>Ok, now that we know how to manage the events at writable stream and readable stream, we can see the Duplex and Transform streams.</p><h4>Duplex and Transform streams, how do they differ from each other?</h4><p>Essentially, Duplex Streams are one Readable Stream with another Writable Stream.</p><p>They have the same behavior as we saw previously, and a Duplex Stream can be used instead of creating two different streams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cDuTzNt_VlOgzjcNWwm_PQ.png" /></figure><p>And about the Transform Stream?</p><p>Let’s suppose that we want to transform our data while using streams. What we can do? We can use a Duplex stream passing data <strong>to each internal</strong> buffer. In theory, we can just connect the data leaving from the Write Stream to the entrance of the Readable Stream, and transform the data while the chunks are flowing between them.</p><p>By doing this, we have our Transform Stream. It’s just it!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vMf52MDGQg5UHDl69hi23w.png" /></figure><p>Great, now that we know the theory, we can start coding something.</p><p>I thought about writing code in each way — writing at first one writable, and then readable, and the duplex/transform — but for educational purposes I will just exemplify with duplex before writing our encrypt/decrypt with transform.</p><h4>Duplex Stream from scratch</h4><p>Our first step will be to declare our Stream structure. As a helper to our implementation, we will extend our main class with DuplexStream from the “streams” package.</p><p>So we got:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c854ee7a1a665e60b55ee0155099c41c/href">https://medium.com/media/c854ee7a1a665e60b55ee0155099c41c/href</a></iframe><p>Now, we should get some properties to initialize our class. For our implementation, we will define the readFilename, the writeFilename, writableHighWaterMark, and the readableHighWaterMark.</p><p>As the name suggests, the “waterMark” properties will be used to define the Buffer’s size of our internal Buffers — as we saw in the last article.</p><p>Our constructor:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fafb09ea148a04c36271f3fca357afd5/href">https://medium.com/media/fafb09ea148a04c36271f3fca357afd5/href</a></iframe><p>Please note:</p><ul><li>The readableHighWaterMark and the writableHighWaterMark are used at super function. As we are extending from the Duplex class, we can set these properties natively.</li><li>For the file names, we’re creating new properties to store them.</li></ul><p>But it’s not only this. How we’re doing our implementation, we need to control the internal chunks and their size.</p><p>So, if the buffer reads a chunk, and then reads another chunk, our “chunkSize” property will be of size two. And the current chunk will be the second one.</p><p>Remember that we’re dealing with streams, so we can’t store all buffers and need to count every chunk that we read.</p><p>Our final constructor will be something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1ddb02bc8953351b45e439fc00cb791/href">https://medium.com/media/b1ddb02bc8953351b45e439fc00cb791/href</a></iframe><p>Ok, to the next step, we need to declare another constructor. But how? Javascript Classes only allow one constructor.</p><p>For streams, we have a method called “_constructor()”. This is our second constructor. I<strong>t’s called right after the original constructor function.</strong></p><p>In this function, we should mount our files and open each source to read and write.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ea4581de9a8de53268445986e6dcce42/href">https://medium.com/media/ea4581de9a8de53268445986e6dcce42/href</a></iframe><p>Please note:</p><ul><li>At this function, we have a callback parameter which is a function.</li><li>This callback function is <strong>responsible for proceeding with the stream flow or notifying the stream that has some error </strong>in our implementation — you can call callback(err) or just callback().</li><li>We’re getting a parameter terminated in “Fd” which means “file descriptor”. File descriptors are the identification of files in our environment. Node uses file descriptors to refer to files in the context of the “fs” module.</li></ul><p>So basically, we’re opening the files that we have declared, and, if some error happens, the stream will be notified about it.</p><p>Our next step will be to implement the write function. As we saw, we have a write buffer inside of the stream and we should use it to manage the flowing chunks.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0112790dd9d093f9ee82ee502b81f1c/href">https://medium.com/media/b0112790dd9d093f9ee82ee502b81f1c/href</a></iframe><p>Please note:</p><ul><li>We’re only using the chunk and callback parameters. The encoding parameter refers to the encoding used in the buffer ( as utf8, ascii, etc..)</li><li>First, we push our current chunk to the chunk property and add the current chunk size to the chunk size property.</li><li>We verify if the current chunk size is higher than our writableHighWaterMark. If yes, this means that our buffer are filled and need to be emptied. So we write to the file, concatenating the existing buffers, verifying if there are no errors, resetting chunk properties, and calling the callback function to proceed with stream flow.</li><li>If not, we just proceed to push the chunk property and call the callback function, waiting for the chunk size to be higher than the writableHighWaterMark.</li></ul><p>Great! Now we already did our write function and need to write our read function, so let’s see how it could be:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f40002677e69db5cd72657a8876909c7/href">https://medium.com/media/f40002677e69db5cd72657a8876909c7/href</a></iframe><p>Note:</p><ul><li>We’re receiving size as a parameter from the read function, and creating a new Buffer to allocate the correct size.</li><li>We’re reading from the reading descriptor (the file that we declared to read) and passing a lot of parameters. These parameters are the <strong>file descriptor,</strong> the <strong>buffer that stores the data</strong> fetched from the file, the <strong>offset position</strong> (initial position from where to start writing), the <strong>length of total data</strong>, the position of <strong>where to start reading</strong>, and the callback with <strong>error and bytes read </strong>quantity.</li><li>We’re pushing the data for the Readable Buffer.</li></ul><p>Not the last, but not the least, the _final function. The _final function is responsible for flushing the last chunk and writing the last time in the Buffer. It’s a simple implementation, but it’s important for the stream lifecycle.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/33626ddf9e781d2e29bb32ea1b01d9bd/href">https://medium.com/media/33626ddf9e781d2e29bb32ea1b01d9bd/href</a></iframe><p>Note:</p><ul><li>We receive a callback function as every function receives.</li><li>We’re resetting the chunks and cleaning our Stream.</li><li>Our callback is called at the end, indicating that our implementation has been finished.</li></ul><p>And now, the least (I promise lol), is the _destroy function. The _destroy function is responsible for the destruction of the stram. It will be executed when:</p><ul><li>The user (or your code) explicitly calls stream.destroy().</li><li>The stream encounters an unrecoverable error internally.</li><li>The stream is being forcibly closed for some other reason (e.g., process shutdown).</li></ul><p>You can handle it in many ways, but for us, I will just ignore it and proceed with the end of the cycle.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a58f85a51bb57ad6575437664b31eaa4/href">https://medium.com/media/a58f85a51bb57ad6575437664b31eaa4/href</a></iframe><p>Great! And now that we have finished our custom stream implementation? We can use it to write and read data! See:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ca20e67dc3b1734eb476e113c336175/href">https://medium.com/media/7ca20e67dc3b1734eb476e113c336175/href</a></iframe><p>Now, if we create a file called “read.txt” and put some string content into it, we will see at the terminal:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2s4ytleBvF0CfXCw8SwVHA.png" /></figure><p>In your created file called “write.txt”, you will see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xgpirNiDCNiL34l_l_HogQ.png" /></figure><p>It’s not well formatted, but it’s ok!</p><p>Now, we know how the process of a duplex stream works, and how to create a custom stream (consider this process the same as creating a readable or writable stream), and we know how to use it in both ways — writing and reading.</p><h4>Finally, Encryption and Decryption Stream!</h4><p>To be honest, maybe the Encryption and Decryption stream could be a disappointment to you. It’s easy to implement them haha.</p><p>All processes are made by the Transform Stream class, and as we saw, we would extend the Transform class in our Encrypt/Decrypt stream.</p><p>But see, we have some advantages here:</p><ul><li>It’s easy to implement</li><li>We are encrypting it on demand. So our hardware impacts are minimal and you can encrypt a huge content!</li></ul><p>Let’s see how we can do it. For the Encrypt Stream:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/db2e036ae7f932f4e8bd4418543317e6/href">https://medium.com/media/db2e036ae7f932f4e8bd4418543317e6/href</a></iframe><p>Note:</p><ul><li>We’re using only the _transform function. This function is responsible for transforming the data and passing it from a writable stream to a readable stream.</li></ul><p>For our “Encryption” — never use this type of encryption. It’s only for educational purposes. If you implement this encryption, easily a hacker can discover and get the decrypted data — we’re using the principle of every time adding one to the Buffer position.</p><p>So if we have a buffer: Buffer&lt;0x01, 0x02, 0x03&gt;, our “encryption” will return a Buffer with: Buffer&lt;0x02, 0x03, 0x04&gt;. Consequently, when we’re using the utf8 decoder, they will interpret as another character.</p><p>Usage notes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2a0dd4a0a84fc7c698712d357ca3ca0a/href">https://medium.com/media/2a0dd4a0a84fc7c698712d357ca3ca0a/href</a></iframe><p>And now, the Decrypt Stream implementation:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3c645e92c0ce87db8000ffb379a68f42/href">https://medium.com/media/3c645e92c0ce87db8000ffb379a68f42/href</a></iframe><p>Note:</p><ul><li>It’s the pretty same thing as the Encrypt Stream, but instead of sum, we’re decreasing in each Buffer position.</li></ul><p>And the usage:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6362cc1e6219587f5f727375f3829705/href">https://medium.com/media/6362cc1e6219587f5f727375f3829705/href</a></iframe><h4>Conclusion</h4><p>If you’ve read this far — and also read the article I mentioned in the first section — I’m sure you understand the implementation.<br>As I always say, my goal here is never to get to the point but rather to explore the possibilities and how to do things from scratch.</p><p>Given our understanding gained from these articles, it is certainly now possible for you to discover more and more about worry-free streams.</p><p>Understand pipelines, async generators in streams, and much more.</p><p>I strongly recommend Erick Wendel’s content about streams.</p><p>In the future, I will talk about networks and how we can create implementations such as TCP and UDP in our applications from scratch, using streams and without external libraries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90a7e5e3bdd2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Writable Streams in NodeJs— Hardware Impact Analysis]]></title>
            <link>https://medium.com/@victorfjansen/writable-streams-in-nodejs-hardware-impact-analysis-bd9002e82a60?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd9002e82a60</guid>
            <category><![CDATA[hardware]]></category>
            <category><![CDATA[node]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Fri, 07 Feb 2025 15:09:41 GMT</pubDate>
            <atom:updated>2025-02-22T03:45:52.246Z</atom:updated>
            <content:encoded><![CDATA[<p>The most efficient way to transfer data between one point in the system and another, using little random access memory (RAM) and taking advantage of computational resources in an intelligent (non-blocking) way.</p><p>At the very least, it is curious how this process works, how buffers are interpreted, how EventEmitters are used and mainly: How the OS and hardware interprets each element in this composition.</p><p>The idea here is to explain what Buffers, EventEmitters are, how writing to files can be done through the node’s “fs” module — the slow way — and how to improve writing speed in addition to optimizing RAM memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O0JrIW_gwGlNfMi-NfwmVA.png" /></figure><p>Make no mistake, everything I’m going to give here is necessary to understand how a Writable Stream works internally. If you’ve ever read anything I’ve written, you know that I like to go into implementation detail.</p><p>Therefore, I recommend reading the session and if you have any questions, I invite you to leave a comment so we can discuss it!</p><h4>After all, what are EventEmitters?</h4><p>An EventEmitter is basically an <strong>object</strong>. An object that intelligently handles the response to specific events.</p><p>The principle used is the observer pattern, which starts listening to an event and, when triggered, an action is executed.</p><p>If you don’t know how an EventEmitter works, here’s an example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce1b7699feb1589a142f207eacad86c0/href">https://medium.com/media/ce1b7699feb1589a142f207eacad86c0/href</a></iframe><p>But why use an EventEmitter?</p><p>Let’s assume we are creating a computer from scratch. Where we are organizing how keyboard clicks will be interpreted by the CPU.</p><p>We can, for example, create a process that will check whether a key has been clicked and, if it has been clicked, the CPU will interpret it as an input signal. That way:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90e86475af93894265e70f762598fa59/href">https://medium.com/media/90e86475af93894265e70f762598fa59/href</a></iframe><p>See:</p><ul><li>We have a process allocated that will check every 100ms if there is an event triggered.</li></ul><p>This is extremely inefficient. When we imagine a CPU communication architecture, we are sure that this process will run on one of the cores, in addition to spending unnecessary RAM on the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BUYLslunQ5hDeHpMCUbG1Q.png" /></figure><p>And this is being triggered all the time. So what alternative do we have?</p><p>Precisely the Observer pattern implemented in EventEmitters. When we use EventEmitter, we are basically structuring an <strong>object </strong>with several callbacks that are called according to need, <strong>without creating a costly process, saving resources due to the very nature of the implementation.</strong></p><p>This way, we have something similar to:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05a1d1b4707e476bb7e856752356cf14/href">https://medium.com/media/05a1d1b4707e476bb7e856752356cf14/href</a></iframe><blockquote>You can see a complete custom implementation created by freecodecamp if you want: <a href="https://www.freecodecamp.org/news/how-to-code-your-own-event-emitter-in-node-js-a-step-by-step-guide-e13b7e7908e1/">https://www.freecodecamp.org/news/how-to-code-your-own-event-emitter-in-node-js-a-step-by-step-guide-e13b7e7908e1/</a></blockquote><p>Please note:</p><ul><li>When we call the <strong>“on”</strong> function, we just add a callback inside the object.</li><li>When the <strong>“emit”</strong> function is called, it has an <strong>O(1)</strong> relationship to find the listeners belonging to that event.</li></ul><p>This way, we don’t have a costly operation as we saw previously and which is executed quickly with several listeners.</p><p>If you are a frontend, I guarantee you that you have already taken advantage of this and don’t remember. An example is the .addEventListener which are added to listen for events emitted by a component in the DOM 🚀.</p><p>I went through this point to explain why implementing the observer pattern is efficient. That’s because our focus<strong> — Writable Stream —</strong> is an EventEmitter.</p><p>This way, if you read everything correctly, you will understand why the implementation is so efficient and uses little memory.</p><h4>Buffers! How do they work?</h4><p>Before talking about Buffers itself, I’m going to go back a little to your first college programming class.</p><p>Remember Binary, Hexadecimal and their conversions that you swore you would never use? I’m sorry to tell you that a Buffer works exactly with these elements.</p><p><strong>Binaries</strong></p><p>As we know, all data that can be interpreted has, at some point, only seen “zeros” and “ones”. Other than that, other measures are different interpretations.</p><p>Any binary — 01000010, for example — can have an interpretation of the same value in hexadecimal — 0x42.</p><p>However, these binaries and hexadecimals, to be understandable to humans, are interpreted by <strong>Encoders</strong>.</p><p>Encoders are conventions adopted to be able to interpret a value — binary, hexadecimal, etc…</p><p>To be clearer, let’s assume we have a large <strong>hexadecimal </strong>number. If I put it into a <strong>character encoder</strong>, I will get several letters that may or may not be random. Whereas if I put it into an <strong>image encoder</strong>, for example, I can get an image of the NASA station.</p><p>All this with the same set of hexadecimal numbers.</p><p>However, let’s just talk about characters, which is what interests us at the moment.</p><p><strong>Character Encoders</strong></p><p>Generally character encoders follow <em>Character Sets</em>. <em>Character Sets</em> are other conventions adopted to be able to interpret, this time, characters! Examples are:</p><ul><li>Unicode (containing 149,813 characters in version 15.1)</li><li>ASCII (which defines 128 characters, uppercase, lowercase, etc.)</li></ul><p>Surely you’ve heard of it and you’ve certainly used it. Each letter in this article, for example, is interpreted as ASCII or Unicode.</p><p>Interestingly, Unicode covers ASCII. So all the characters in ASCII are the same as in Unicode (same binary values), but Unicode covers many other types of alphabets in addition to ASCII.</p><p>So, if we type the character “B” (in capital letters), I know that its binary value is 01000010 and its hexadecimal value is 0x042.</p><p>Both Unicode and ASCII have a table in which both will reflect the character “B”.</p><p>Then:</p><ul><li>I have the hexadecimal code: 0x042</li><li>Step into an Encoder that complies with the Unicode standard</li><li>The result I got is “B” (capital b)</li></ul><p>It is also important to mention that the Unicode encoder is <strong>“UTF-8”</strong> (being used most of the time — it is even the default for js/node) and the ASCII encoder is “ascii”.</p><blockquote>This was a brief explanation for those who already have an understanding of how encoders and unicode work, however, if you are interested, you can read more about: <a href="https://www.ime.usp.br/~pf/algorithms/appendices/unicode.html">https://www.ime.usp.br/~pf/algorithms/appendices/unicode.html</a></blockquote><p><strong>Finally, Buffers!</strong></p><p>Now that we have an understanding of how binaries are interpreted and how encoders work, we can get back to Buffers.</p><p>Essentially, what is a Buffer?</p><p>Think of it as a variable. Just a temporary variable that is generally used for data transfer.</p><p>But this variable counts and considers bytes. So all internal operations related to Buffers are directly related to the count of existing bytes<strong> — remembering that 8 bits is equal to 1 byte.</strong></p><p>Furthermore, it also considers data storage in hexadecimal. This is why conversion becomes important!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/1*Eoy5XKhz8ZJ_AFdEPWlk1A.png" /></figure><p>An example to make it clear → Let’s assume we have the following sequence of bits:</p><ul><li>0100 1000 0110 1001 0010 0001</li></ul><p>And I tell you: these bits form a word. How can we do this transformation with Buffers in Node?</p><p>Simple! Just allocate the correct amount, convert to hexadecimal and use the UTF8 encoder to find the word!</p><p>That way:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df470432b87548f023a5aa3f6014b81a/href">https://medium.com/media/df470432b87548f023a5aa3f6014b81a/href</a></iframe><p>Please note:</p><ul><li>I used every two sets of 4 bits (totaling 8 bits) to convert to hexadecimal.</li><li>The result is “Hi!” — printed on the console.</li></ul><p>So, I allocate a Buffer and its hexadecimal values ​​and interpret them with an encoder.</p><blockquote>NOTE: never allocate your buffers this way. It was just an example to understand the byte positions.</blockquote><p>Thinking about efficiency, we have nothing new here. As I said, a Buffer is a variable that temporarily allocates memory.</p><p>If I allocate a buffer, for example, that consumes a number of bytes equivalent to 5GB, my RAM memory will allocate that entire amount and occupy the system with something that is not necessarily useful.</p><p>Example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f50dd2891802da064b7c3dca5d06afbf/href">https://medium.com/media/f50dd2891802da064b7c3dca5d06afbf/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yToZ4UBQOECT7t942nNjcw.png" /></figure><p>Please note:</p><ul><li>5e9 bytes is an approximate value of 5GB</li><li>The memory reserved for this allocation process was 4811M, equivalent to 4.8GB.</li><li>I’m wasting memory for nothing. There is no need to use all this memory by far.</li></ul><p>Now I believe we have a basis to proceed with the implementation of Writable Streams.</p><h4>Don’t read if you think it’s a waste of time — just curious about the node’s internal allocation</h4><p>In Buffers, we have some interesting methods such as:</p><ul><li>Buffer.alloc()</li><li>Buffer.allocUnsafe()</li><li>Buffer.allocUnsafeSlow()</li></ul><p>How do these methods directly relate to memory allocation? Secure Allocation? Insecure? Slow? Fast?</p><p>The Node itself, when it starts its process, has a pre-allocated memory behind the scenes to perform operations.</p><p>This memory is calculated through a constant in Buffer called <strong>“poolSize” — by default 8 Kib.</strong></p><p>note: 8 Kib!== 8KB. 8Kib = 8 * 1024 bytes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ezTKUnIpvP2Tnpb6RgX2FA.png" /></figure><p>The memory allocated by poolSize, among other reasons for its existence, can be used to allocate Buffers that the developer wants!</p><p>However, we have a detail: Only half of the poolSize can be used to allocate Buffers. Internally a byteshift operation is performed and reduces this “poolSize” value by two times. Logo Buffer.poolSize &gt;&gt;&gt; 1;</p><p>You can check this value using:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7cbd78a981c34c6f80a654b32d2e692/href">https://medium.com/media/d7cbd78a981c34c6f80a654b32d2e692/href</a></iframe><p>When we use the “Buffer.alloc()” method, we are making a safe allocation. This means that we do not use this reserved memory and under the hood we are allocating our entire Buffer in a new memory segmentation.</p><p>Now when we use “Buffer.allocUnsafe()” we use pre-allocated memory. And in fact, it is a faster allocation because it is already reserved memory. Now, <strong>we have to be careful</strong> because there may be data in this already allocated memory. So if it is a sensitive application, it is possible to print the allocated memory and see the bytes in it. It’s not a guarantee, but information that should not be shared there may be found, in addition to overwriting this segmentation!</p><p>You can test with:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3e6e6565c7f19c8c228457258ab61309/href">https://medium.com/media/3e6e6565c7f19c8c228457258ab61309/href</a></iframe><p>In fact, while writing this article, I managed to find a piece of old code inside this buffer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X-_neE6-lNrSpe8eRGDgNw.png" /></figure><p>Now, in relation to “Buffer.unsafeSlow()”, it is the same idea as “Buffer.unsafe()”, but not using pre-allocated memory. Either way, it’s unsafe.</p><h4>Writable Streams</h4><p>Now, let’s get to the point.</p><blockquote>I will present the scenarios and then the theory!!</blockquote><p>Before starting, I will use the file writing scenario to exemplify writable streams.</p><p>It’s a good example that can cause a lot of headaches, allocated time and memory lol.</p><p><strong>Our main problem</strong></p><p>Let’s assume we want to write 1 million logs to a simple “.txt” file.</p><p>We have this task and we want to complete this writing in the shortest possible time and using the least possible resources, both CPU and RAM.</p><blockquote>For all scenarios, we will measure performance in milliseconds and compare RAM.</blockquote><p>How do we do it?</p><p>Let’s look at the options:</p><ol><li>We can do this writing with fs/promises (asynchronous fs module). Let’s open the file reading, create some random sentence and add this writing. This way we have:</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e5d248bf94f17201921932124f2aad77/href">https://medium.com/media/e5d248bf94f17201921932124f2aad77/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y0qlfEWQrwAT1_mFZm1UVA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6LHlnsKeykY_Pn9Xn4svCQ.png" /></figure><p>Please note:</p><ul><li>At peak execution, we have something around 50MB of RAM being used.</li><li>The time to complete the writing was 34564ms, which is 34.5 seconds! Pretty slow to tell the truth.</li><li>The CPU gones well. 48%.</li></ul><p>It’s a good scenario where we save memory, but it’s still very slow. If it’s a process that fits in a queue, for example, it’s a very bad scenario. I’m considering logs of up to 1 million, but what if there happen to be more?</p><p>So let’s go to option 2:</p><p>2. Still with fs/promises, but with synchronous writing!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/72c9f99de335bec95d00f9e8f232e3cc/href">https://medium.com/media/72c9f99de335bec95d00f9e8f232e3cc/href</a></iframe><p>An interesting fact is that the memory exploded!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N3RJWfosIIfUniWhLya3nA.png" /></figure><p>However, I’m going to increase it so we can see how far it goes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VVZv7Z_2CNbPnZ7mp_-vTA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d3YK5seu4AEiRwhDuA_wjA.png" /></figure><p>Please note:</p><ul><li>It was faster! 23848 ms, which is 23.4 seconds!</li><li>The RAM memory was occupied with almost 4GB! Much more than the other asynchronous process!</li><li>The CPU is in 94%! Much more than the other asynchronous process!</li></ul><p>It’s a good result, but I believe that no one has infinite computing resources to be able to store all this RAM.</p><p>So let’s go to option 3:</p><p>3. With the fs module (synchronous) and callbacks!</p><p>In the Node documentation, the team mentions that callbacks have better performance. So don’t be surprised by the results.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc32d4c84a88c1b32b91c9203fadb29c/href">https://medium.com/media/bc32d4c84a88c1b32b91c9203fadb29c/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UQ9R3TM2wBqsOd6Aahgfng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAw7ty1LoXCf5MSThED9Ew.png" /></figure><p>Please note:</p><ul><li>Time has reduced a lot! 799ms, which is 0.7 seconds!</li><li>Excellent RAM memory! 50MB of usage!</li><li>CPU in only 11% of usage!</li></ul><p>This is the best scenario so far! Which also proves that using callbacks are much more performant than promises.</p><p>But we can still improve!</p><p>Let’s go to option 4:</p><p>4. Writing to files with streams! 🚀</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3455b5166b4edb3095935e5d291126e1/href">https://medium.com/media/3455b5166b4edb3095935e5d291126e1/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7BA_hSj5aeYkOpL0nnrFRA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l4IX1e5l0AUbclu77tktEw.png" /></figure><p>Realize:</p><ul><li>It was just 234ms, which is 0.2 seconds! Incredibly performative!</li><li>The CPU in only 2%! Really performative!</li><li>Has the RAM memory increased? Wasn’t the stream more performative?</li></ul><p>After all this theory about Buffers, binaries, encoders, EventEmitters, does the stream still consume more memory than a simple callback?</p><p>No! This last implementation is wrong!</p><p>I put it on purpose so we can understand the problem and how Writable Streams work under the hood.</p><p><strong>Let’s get to the theory:</strong></p><p>A Writable Stream is essentially a set of EventEmitters and Buffers. Basically the idea is:</p><ul><li>have a Buffer that will fill itself with data</li><li>After it is full, it writes this data</li><li>As soon as it is emptied, it is filled again with new data</li></ul><p>And so on until the stream ends. That way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qNLGkmJAmY9rvvsluO6-wA.png" /></figure><p>Please note:</p><ul><li>The internal buffer size of a writable stream is 16384 Bytes. This means that with each fill, only this value can be filled.</li><li>In addition to the buffer, there are events and properties in the stream. Where the “events” are from EventEmitter.</li></ul><p>To know the internal size of the buffer, you can look for a property in the stream called <strong>writableHighWaterMark</strong>. It indicates the maximum size of the existing Buffer.</p><p>Now, how do I know how much I’ve already written in this Buffer? You can search for the <strong>writableLength </strong>property.</p><p>This way:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c4c00d4406634b059021993d01c5dac/href">https://medium.com/media/6c4c00d4406634b059021993d01c5dac/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3cPjvsArS9BFYLZCAftWaQ.png" /></figure><p>Realize:</p><ul><li>Is the initial stream limit 65536? Hadn’t I mentioned that it was 16384 Bytes? Yes. For streams created from the “stream” module, they are 16384. However, for the “fs” module — file open streams — , the streams are created with 64Kib (Node version 22.13.0).</li><li>The initial written value is 0.</li><li>The value written after the phrase “My text!” is 8. Essentially because each character is made up of 8 bits and for every 8 bits we have 1 byte. So 8 * 8 / 8 = 8 Bytes.</li></ul><p>This way, the buffer is filled and its internal properties reflect its real value.</p><p>But what if the padding is larger than the Buffer size? <strong>This is the point we want!</strong></p><p>When the Buffer content is larger than expected by the Stream, this value remains in “memory”. In other words, a new allocation is made under the table to store this “leftover” until the Buffer is emptied and filled again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kJZ1-7fhN4CYyK5VNcC4XA.png" /></figure><p>And this is what happens in our previous implementation (impl 4)! We are filling the stream’s internal Buffer, but there is some data left to be filled. This way, more RAM is unnecessarily allocated, consuming more in the writing process.</p><p>And how do we get it right then? Let’s go to our 5 implementation:</p><p>5. Using streams the right way!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f849a65d8ef1b861cd0185b328a08349/href">https://medium.com/media/f849a65d8ef1b861cd0185b328a08349/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rNFjEcaZN2En3lioApoL2Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iSKYljlMUoTRjmt2dvatAg.png" /></figure><p>We have a lot of things to commit here:</p><ul><li>The weather remains average! However, based on my tests, it still takes around 10ms to 20ms longer.</li><li>Memory was cut in half. Only 41MB at this writing. Compared to the previous result, a good cut.</li></ul><p><strong>Regarding the code:</strong></p><p>We now use a while() loop placing the control variable outside the loop. This way I can control my executions and continue when I have to.</p><p>The first stopping condition (i === MAX_ITERATIONS — 1) is the maximum number of possible iterations, where we must close the stream so as not to write more than expected. Therefore, stream.end(buffer).</p><p>The second stopping condition refers to writing the stream! As we mentioned and did before, the stream.write() method writes the value of the current buffer to the stream. However, one thing I didn’t mention is: the .write() function returns a boolean value! This Boolean value refers to the Buffer limit.</p><p>If the written value (<strong>writableLength</strong>) is greater than or equal to the limit value (<strong>writableHighWaterMark</strong>), the emission becomes “false”, which triggers the while loop to finish.</p><p>Soon after we started hearing the “drain” and “finish” events. Remember I explained how EventEmitters work? That’s what it was for. Simply streams are also EventtEmitters.</p><p>Each stream has some custom events. In our case, we have:</p><ul><li><strong>Drain: </strong>means the stream has been emptied</li><li><strong>Finish: </strong>means the stream has ended</li></ul><p>This way, the stream fills, breaks the <strong>while </strong>loop, writes the data from the internal buffer, empties it, and the<strong> “drain” event is triggered</strong>. And so on until the stream is closed.</p><p>When closed, closes the file opening.</p><h4>Conclusion</h4><p>I believe you have managed to understand the fundamentals of writable stream and the impact of its use on a daily basis.</p><p>My goal here is not to be direct and try to develop more critical thinking in relation to use, showing different use cases and how a poorly done implementation can cost you dearly💸.</p><p>The codes related to the stream itself are available in this repository:</p><p><a href="https://github.com/victorfjansen/writable-streams">GitHub - victorfjansen/writable-streams</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd9002e82a60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to obfuscate your JS bundles — Protect your frontend applications]]></title>
            <link>https://medium.com/@victorfjansen/how-to-obfuscate-your-js-bundles-3533bb52ce12?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/3533bb52ce12</guid>
            <category><![CDATA[node]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[obfuscation]]></category>
            <category><![CDATA[cryptography]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Tue, 28 Jan 2025 12:00:50 GMT</pubDate>
            <atom:updated>2025-01-28T13:19:22.874Z</atom:updated>
            <content:encoded><![CDATA[<p>Surely you have already done some process on the frontend that needed to be protected. I mean, some image processing, PDF, or file itself (however common it is to do it on the backend lol). I believe that, to avoid code exploitation and increase the level of protection of your application, obfuscating JS bundles can be perfectly acceptable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tnb5FwHpIiCtIIzZPcTxiA.png" /></figure><h4>After all, what is obfuscation?</h4><p>Obfuscation is used to make code difficult to read, as well as protecting it against malicious manipulation and reverse engineering attempts.</p><p>Let’s assume you have a webworker that processes certain data (such as images) and needs to ensure integrity or minification before sending it to the backend. Or perhaps, there is some mimetype checking and sanitization of the input data. If by chance someone wants to manipulate it maliciously, they will have access to all the code.</p><blockquote>Obfuscation does not necessarily mean encryption, but one case does not exclude the other. It is possible to combine both strategies to keep the code secure.</blockquote><p>Here are some key points:</p><p><strong>Advantages of obfuscation:</strong></p><ul><li>Makes it difficult to exploit vulnerabilities;</li><li>Intellectual property protection;</li><li>Prevention of unauthorized modifications:</li><li>Prevention of unauthorized modifications:</li></ul><p><strong>Disadvantages:</strong></p><ul><li>Increase in bundle size (It will not necessarily increase, however there are strategies that impact the final size);</li><li>Difficulty debugging;</li><li>Possible performance impact (depending on whether adding encryption, for example, will add performance impacts);</li></ul><p>But given the conditions, we can at least change, for example, the name of the variables to hexadecimal. This will help with obfuscation as well as other techniques.</p><h4>Adopted Strategy</h4><p>Today we have two options:</p><ul><li>Configure our application builder with webpack and add obfuscation.</li><li>After build, regenerate each chunk.js adding our obfuscation.</li></ul><p>I will adopt the <strong>second strategy</strong> as I believe it is more flexible. Not necessarily every application will be built with webpack. Many more modern applications, for example, use Vite to build (which is infinitely faster) and can use this implementation.</p><p>Additionally, I will use angular in my examples, but this technique will certainly work for other builds of other frameworks. After all, it’s all JS.</p><h4>How do we start?</h4><p>The first step is to install our obfuscation library. We will use the javascript-obfuscator .</p><p>Be with NPM:</p><pre>npm install javascript-obfuscator -D</pre><p>or with Yarn, for example:</p><pre>yarn add javascript-obfuscator -D</pre><p>The second step is to configure a file in your root directory.</p><p>Create a file called “javascript-obfuscator.config.json”. In this file, we will pass the flags that we want to configure our obfuscation.</p><blockquote>You can check the available options on the lib website. The descriptions even detail the risks and slowdowns that each property can generate. You can consult here: <a href="https://obfuscator.io/">https://obfuscator.io</a></blockquote><p>For our configuration, I will use a set of configurations that I even use in production. The build size increases (due to dead code injection — we’ll see later), but I lose almost nothing in performance.</p><p>So, creating the file javascript-obfuscator.config.json :</p><pre>{<br>    &quot;compact&quot;: true,<br>    &quot;deadCodeInjection&quot;: true,<br>    &quot;stringArray&quot;: true,<br>    &quot;stringArrayEncoding&quot;: [&quot;none&quot;],<br>    &quot;stringArrayThreshold&quot;: 0.5<br>}</pre><p>Where:</p><ul><li><strong>compact: </strong>Compact code output on one line.</li><li><strong>deadCodeInjection</strong>: With this option, random blocks of dead code will be added to the obfuscated code (just a curiosity: dead-code-injection increased 4MB in the final build — the documentation mentions an increase of up to 200% in more severe cases!).</li><li><strong>stringArray</strong>: Removes string literals and place them in a special array. For instance, the string “Hello World” in var m = “Hello World”; will be replaced with something like var m = _0x12c456[0x1];</li><li><strong>stringArrayEncoding</strong>: encode all string literals of the <a href="https://obfuscator.io/#stringarray">stringArray</a> using base64 or rc4 and inserts a special code that used to decode it back at runtime — <strong>it can slow down your application if setted as base64 or rc4.</strong></li><li><strong>stringArrayThreshold</strong>: ou can use this setting to adjust the probability (from 0 to 1) that a string literal will be inserted into the stringArray <strong>— it can slow down your application if value is high.</strong></li></ul><blockquote>It is important to mention that <strong>some options are activated by default (even if they are not even mentioned in this .json).</strong> Therefore, I recommend reading the documentation to be aware of possible changes. — <a href="https://obfuscator.io/">https://obfuscator.io</a></blockquote><p>This way, by installing the dependency and generating the configurations in the .json, we can create a new script in the package.json!</p><p>So we will have:</p><pre>&quot;scripts&quot;: {<br>  ...other_scripts,<br>  &quot;build:obfuscated&quot;: &quot;ng build &amp;&amp; javascript-obfuscator &lt;PATH_TO_YOUR_BUILDED_CONTENT&gt; --output &lt;PATH_TO_YOUR_BUILDED_CONTENT&gt; --config javascript-obfuscator.config.json&quot;,<br>}</pre><p>Note that you must add the path of your generated build next to the script. As a result, every file in this directory will remain with the same name, but with the content obfuscated.</p><h4>Final Tip</h4><p>You can consult ready-made presets on the documentation website. You can find presets like:</p><ul><li>High obfuscation, low performance</li><li>Medium obfuscation, optimal performance</li><li>Low obfuscation, High performance</li><li>Default preset, High performance</li></ul><p>That way, you can test and see how your application reacts!</p><h4>Conclusion</h4><p>If you see the need to add encryption in addition to obfuscation, just configure it in the .json file!</p><p>Your will here is your limit! (and the loading speed too lol)</p><p>Be careful not to significantly affect your application and remember: it will always be a trade-off — some benefits for other harms.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3533bb52ce12" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Managing Angular with Kubernetes (Canary deployments with ArgoRollouts)]]></title>
            <link>https://medium.com/@victorfjansen/managing-angular-with-kubernetes-canary-deployments-with-argorollouts-97be2b706a0f?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/97be2b706a0f</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[canary-deployments]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[argo-rollouts]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 05:42:03 GMT</pubDate>
            <atom:updated>2024-10-07T05:43:35.071Z</atom:updated>
            <content:encoded><![CDATA[<p>Um dos principais aspectos do deploy em produção é a entrega progressiva e o monitoramento de recursos. Isso é especialmente relevante em ambientes de backend, mas e quando se trata do frontend?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EwOUTZx0JdO3mrPNsUSNcQ.png" /></figure><blockquote>Importante ressaltar que esse artigo foi escrito apenas para trazer a ideia do canary deployment. São vários pontos que exigem certo conhecimento. Se for falar de cada detalhe, irá ficar enorme. Por esse motivo, sempre haverá links nas sessões com mais complementos.</blockquote><p><strong>TLDR;</strong></p><p>A prática de <strong>entrega contínua</strong> e o uso de métodos de validação são importantes em produtos com um público grande. Mas e se uma nova feature apresentar problemas em uma determinada implementação? Como garantir uma reversão rápida? Ou, se for necessário validar uma hipótese, como redirecionar parte dos usuários para uma nova versão sem impactar a experiência de todos?</p><p>O <strong>Canary deployment</strong> permite um caminho para esses cenários, porque possibilita o lançamento de novas versões de uma aplicação para uma parcela específica de usuários. Assim, minimizamos riscos, pois a versão nova é testada em produção de forma controlada. Além disso, facilita o monitoramento de desempenho e o feedback dos usuários de modo granular.</p><p>Esse tipo de deployment pode ser ainda mais interessante. Com o Canary, podemos avaliar a receptividade dos usuários às novas interfaces, detectar problemas específicos que possam surgir em diferentes navegadores ou dispositivos, e até mesmo ajustar elementos de usabilidade com base em dados reais de interação. Como isso impacta a experiência do usuário? Será que a nova versão realmente melhora o engajamento?</p><h4>Vantagens e desvantagens: Não seria melhor usar uma CDN?</h4><p>Quando consideramos a entrega de conteúdo e a implementação de escalabilidade em aplicações modernas, é comum a dúvida: por que não utilizar uma <strong>Content Delivery Network (CDN)</strong> ao invés de uma estrutura em <strong>Kubernetes</strong>? Cada tecnologia tem seus pontos fortes e fracos, e a escolha ideal depende das necessidades específicas do projeto.</p><h4>Vantagens de usar uma CDN:</h4><ol><li><strong>Redução da Latência:</strong> A CDN armazena recursos estáticos em servidores distribuídos globalmente, o que permite que os dados sejam entregues a partir do local mais próximo do usuário, reduzindo o tempo de resposta e melhora a experiência do usuário final.</li><li><strong>Facilidade de Escalabilidade para Conteúdo Estático:</strong> Para aplicações que lidam com alto tráfego de conteúdo estático (imagens, vídeos, scripts, etc.), uma CDN pode atender a um grande volume de requisições sem sobrecarregar os servidores principais.</li><li><strong>Alta Disponibilidade:</strong> CDNs são projetadas para garantir a entrega contínua, mesmo em casos de falha de servidor. Elas contam com mecanismos de redundância que garantem a disponibilidade do conteúdo.</li><li><strong>Segurança Adicional:</strong> Além da entrega de conteúdo, muitas CDNs incluem proteção contra ataques DDoS e suporte a certificação SSL, adicionando camadas de segurança sem a necessidade de configurações adicionais no ambiente Kubernetes.</li></ol><h4>Desvantagens de usar uma CDN:</h4><ol><li><strong>Menos Controle sobre Versionamento e Lançamentos:</strong> CDNs não possuem a mesma flexibilidade que o Kubernetes para gerenciar versões diferentes da aplicação. Em um ambiente Kubernetes, é possível adotar o <strong>Canary deployment </strong>por exemplo, lançando atualizações gradualmente e monitorando o impacto em tempo real.</li><li><strong>Necessidade de Gerenciar o Cache:</strong> Enquanto o cache em CDNs melhora o desempenho, também pode ser um ponto difícil quando é necessário invalidar ou atualizar o conteúdo de forma rápida. No Kubernetes, a atualização dos pods e o uso de balanceadores de carga internos permitem que o conteúdo seja entregue sem preocupações com invalidações de cache.</li><li><strong>Menos controle sobre ambiente e integração com serviços externos: </strong>O Kubernetes oferece um controle detalhado sobre o ambiente, permitindo a criação de configurações personalizadas, testes de versões específicas, e a integração com diversos serviços externos, já a CDN, dependendo do provedor, não fornecerá recursos o suficiente a depender da integração desejada.</li></ol><p>No geral, tudo depende dos requisitos do seu projeto e da arquitetura definida. Cabe a você decidir o melhor caso.</p><h4>Pré Requisitos</h4><p>Considerando o tema, tenha em mente que precisará de conhecimentos básicos sobre:</p><ul><li>Docker (<a href="https://www.freecodecamp.org/news/how-docker-containers-work/">https://www.freecodecamp.org/news/how-docker-containers-work/</a>)</li><li>Kubernetes (<a href="https://blog.gruntwork.io/a-crash-course-on-kubernetes-a96c3891ad82">https://blog.gruntwork.io/a-crash-course-on-kubernetes-a96c3891ad82</a>)</li><li>Angular (<a href="https://loiane.training/curso/angular">https://loiane.training/curso/angular</a>)</li><li>NGINX (<a href="http://nginx.org/en/docs/beginners_guide.html">http://nginx.org/en/docs/beginners_guide.html</a>)</li><li>Canary deployments (<a href="https://semaphoreci.com/blog/what-is-canary-deployment">https://semaphoreci.com/blog/what-is-canary-deployment</a>)</li></ul><h4>Um pouco da teoria: Canary Deployment</h4><p>O Canary deployment não é nada mais que uma estratégia para garantir a escalabilidade, reduzindo os riscos de um deploy ou permitindo a validação de uma feature com um grupo seleto de usuários.</p><p>No Canary Deployment, uma nova versão do software é lançada para um grupo de usuários, chamado de “grupo canário.” Esses usuários são expostos à nova versão, enquanto a maioria dos usuários continua usando a versão anterior. Isso permite a equipe de desenvolvimento monitorar o desempenho, a estabilidade e a experiência do usuário em uma escala reduzida.</p><p>Se não forem identificados problemas significativos, a nova versão é gradualmente ampliada para mais usuários até que todos estejam usando a versão atualizada. No entanto, se forem encontrados erros ou impactos negativos, a equipe pode facilmente reverter a mudança para a versão anterior, minimizando o impacto sobre o sistema como um todo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VYhmIeIQwEin-c3oUHRYZQ.gif" /><figcaption>GIF from <a href="https://octopus.com/devops/software-deployments/canary-deployment/">https://octopus.com/devops/software-deployments/canary-deployment/</a></figcaption></figure><p>Dado esse exemplo, podemos integrar com o Kubernetes e garantir que o nosso deploy de novas versões irá ficar estável. O que teremos que fazer é o controle de imagens dentro dos pods. <br>Para nos ajudar com essa tarefa, iremos usar os ArgoRollouts e seu painel iterativo, permitindo uma visualisação clara dos processos com um dashboard iterativo.</p><h4>Mãos na Massa! Criando nossa aplicação Angular!</h4><p>Bom, para começar, não tem muito segredo haha! Temos que criar nossa aplicação Angular.</p><p>Com ela, vamos seguir o seguinte fluxo para seguirmos o deploy em nosso cluster Kubernetes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YCUvGdUDPSM5wQ_ykZLcOQ.png" /></figure><p>Tendo a CLI do angular instalada em sua máquina e seguindo as instruções do<a href="https://angular.dev/cli/new/"> <strong>guide oficial</strong></a><strong>,</strong> usamos:</p><pre>ng new angular-app-kube</pre><p>Assim:</p><ul><li>Ao rodarmos npm run start teremos a nossa aplicação:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wtIDBnZDzdtjsc6h6CSxDQ.png" /></figure><p>Note que o &lt;h1/&gt; dela contém “THIS IS THE SECOND VERSION”. Iremos alterar e balancear a carga para uma outra aplicação posteriormente que terá outra escrita. Lembrre-se de memorizar!</p><blockquote>Importante se atentar para criar essa estrutura (vamos fazer as operações de build dentro da aplicação Angular):</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*GDg4239CBNKMoScqe1TDXQ.png" /></figure><p>Ok, agora que criamos a nossa aplicação, vamos montar a imagem dela.</p><p>Portanto, na raiz do diretório, crie um Dockerfile para podermos buildar a nossa imagem.</p><p>Assim, teremos:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc73f155d1110b84d2788a9c3a17c06b/href">https://medium.com/media/bc73f155d1110b84d2788a9c3a17c06b/href</a></iframe><p>Perceba:</p><ul><li>Fazemos o install e build dentro do docker</li><li>Usamos um multistage build para pegar apenas os arquivos que precisamos (vulgo “build”) e jogamos para dentro da /usr/share/nginx/html. Os arquivos, assim que o conteiner subir, serão servidos automaticamente.</li><li><strong>Você pode adicionar uma configuração própria para o seu NGINX. Tratamentos de cache e entre outros podem ser feitos para aprimorar o delivery do seu conteúdo.</strong></li></ul><p>O build com a configuração inicial será o vfjansen/angular-app-kube:v2 .</p><p>Da mesma maneira, geraremos um segundo com uma outra label, alterando o arquivo app.component.html .</p><p>Assim:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uTRviim3geLNTFhTOylzBw.png" /></figure><p>Perceba que nessa outra versão, vfjansen/angular-app-kube:v9 , tem um outro texto contendo “THIS IS THE OTHER VERSION HEEEERE”.</p><p>Iremos comparar essas labels quando configurarmos o balancer e ver o canary ocorrendo em tempo real.</p><p>Depois das imagens criadas e buildadas, <strong>**lembre-se de enviar essas imagens ao docker hub com </strong><strong>docker push image_name .</strong></p><h4>Criando nosso cluster e configurando o Kubernetes</h4><p>O primeiro passo para criarmos nosso cluster será <a href="https://kubernetes.io/docs/tasks/tools/">configurar o Kubectl. </a>Basicamente, ele é o responsável por comunicar com os nodes dentro dos clusters.</p><p>Após a configuração do Kubectl, temos que usar alguma ferramenta como o <a href="https://kind.sigs.k8s.io/">Kind</a> ou o <a href="https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/">Minikube</a> para provisionar um ambiente Kubernetes local. <br>Essas ferramentas facilitam a criação de clusters Kubernetes em máquinas pessoais ou ambientes de desenvolvimento, sem a necessidade de infraestrutura em nuvem.</p><p>Como costume, criarei nosso cluster com o <a href="https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/">Minikube</a> por provisionar o cluster com apenas um comando.</p><p>Portanto, caso não tenha nenhuma ferramenta para o provisionamento, procure <a href="https://minikube.sigs.k8s.io/docs/start/?arch=%2Fwindows%2Fx86-64%2Fstable%2F.exe+download">instalar nos sites oficiais.</a></p><p>A estrutura do cluster que o Minikube cria, a mérito de arquitetura, envolve o controle plane e os nodes. Para o que precisamos no momento, é o suficiente (certamente outras peças são criadas, porém vou abstrair pra mantermos o foco).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gsKwRJRJ3c6w0kn2I4-AhA.png" /><figcaption><a href="https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/">https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/</a></figcaption></figure><p>Assim:</p><ul><li>As nossas operações se concentrarão em configurar services, deployments e ingresses controladas pelo control-plane. Os nodes receberão as instâncias dos Pods e trarão a escalabilidade que precisamos.</li></ul><p>Para criarmos com o cluster com o minikube, devemos abrir um terminal inputar minikube start :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ll5KstEZtg1L1LTe1mOavA.png" /></figure><p>Pronto! Assim criamos nosso cluster. Para verificar o status, basta digitar:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IanbDZpaL4nF4eLdHPkZ1Q.png" /></figure><p>Perfeito! Agora, voltando a nossa aplicação, iremos criar o nosso primeiro deployment.</p><blockquote>Vou deixar um guia caso não saiba como deployments funcionam:</blockquote><p><a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployments</a></p><p>Usando a imagem que montamos anteriormente, vamos seguir subindo no nosso cluster a nossa primeira imagem. No diretório k8s criaremos o arquivo deployments.yml (seguir a estrutura apresentada no início).</p><p>Assim:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b53af2e7cebbca79dce51cd8f8606156/href">https://medium.com/media/b53af2e7cebbca79dce51cd8f8606156/href</a></iframe><p>Perceba:</p><ul><li>Estamos usando 5 replicas para a nossa aplicação</li><li>Limitamos os recursos para cada POD (como nossa aplicação é leve, não estou considerando nenhum cálculo de carga, apenas valores mínimos que aguentarão rodar a nossa aplicação).</li><li>Estamos usando o startupProbe para verificar o início da disponibilidade do POD.</li><li>Estamos usando o LivenessProbe para, caso dê algum problema, reiniciar o nosso POD.</li><li>Estamos usando o ReadnessProbe para redirecionar o trafego dos PODs para outros enquanto não estiverem disponíveis.</li></ul><p>Antes de subirmos o nosso Deployment , é interessante criarmos um service e um ingress.</p><p>Basicamente, uma <strong>Service </strong>é um ponto de entrada para o deployment, onde roteia o tráfego para os PODs gerenciados pelo deployment, considerando o balanceamento de carga.</p><p>Um <strong>Ingress</strong> é um recurso que gerencia o acesso externo aos serviços do cluster, geralmente usando protocolos HTTP e HTTPS. Ele fornece regras de roteamento para direcionar o tráfego de entrada para os serviços apropriados com base em parâmetros como o caminho da URL ou o nome do host.</p><p>Dessa forma conseguimos controlar o tráfego interno e externo para o redirecionamento aos PODs.</p><p>A nossa Service:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/66c2df9f53126f98f571bcab1d69eb5f/href">https://medium.com/media/66c2df9f53126f98f571bcab1d69eb5f/href</a></iframe><p>Perceba:</p><ul><li>O selector aponta para o nome do nosso Deployment, onde fara a ponte entre os acessos e os estáticos fornecidos pelo POD.</li><li>Estamos usando o ClusterIP (como teremos o ingress, o IP externo será configurado, redirecionado à service)</li><li>Apontamos para a porta 80, onde o nginx roda por padrão.</li></ul><p>Para a nossa Ingress, vamos utilizar o <strong>Ingress-NGINX</strong>. O Ingress-NGINX é um controller que monitora recursos de Ingress associados a ele. No nosso caso, vamos criar um recurso de Ingress usando o controller Ingress-NGINX para redirecionar os acessos externos ao nosso serviço.</p><p>Para mais detalhes sobre controllers:</p><p><a href="https://able8.medium.com/kubernetes-controllers-overview-b6ec086c1fb">Kubernetes: Controller Overview</a></p><p>É importante também configurar o ingress-nginx no cluster. Você pode aplicar o manifesto do nginx. Siga o passo a passo. Após configurar, verifique se a controller está rodando com kubectl get pods -n ingress-nginx .</p><p><a href="https://docs.k0sproject.io/v1.27.1+k0s.0/examples/nginx-ingress/">NGINX Ingress Controller - Documentation</a></p><p>Assim:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51b6cfdf36a4411e52d6caae0a2eba0a/href">https://medium.com/media/51b6cfdf36a4411e52d6caae0a2eba0a/href</a></iframe><p>Perceba:</p><ul><li>Nas annotations usamos o rewrite-target, onde, sempre que batemos na ingress, será redirecionado pro root. Exemplo: se eu tentar acessar http://&lt;ingress-host&gt;/alguma-rota, serei redirecionado para http://angular-canary/ .</li><li>Nas Specs, usamos a ingressClassName como NGINX. Assim, definimos um path que direciona o tráfego HTTP para um serviço específico dentro do cluster, permitindo que a aplicação responda a solicitações na URL definida. Portanto, toda vez que a ingress receber o path “/”, será redirecionado à service angular-canary (como foi definido no backend).</li></ul><h4>Subindo nossos PODs</h4><p>Agora que finalizamos a criação dos arquivos. Use o Kubectl para aplicar as configurações e criar os deployments, services e ingress.</p><p>Assim, rode no seu terminal apontando para o seus arquivos:</p><pre>kubectl apply -f k8s/service.yml<br>kubectl apply -f k8s/ingress.yml<br>kubectl apply -f k8s/deployment.yml</pre><p>Você pode consultar os serviços com:</p><pre>kubectl get deployments<br>kubectl get pods<br>kubectl get ingress</pre><p>Todos devem estar rodando e os pods devem estar no estado de Ready! Assim, sabemos que estão prontos para receber tráfego e rotear a aplicação angular.</p><p>Porém, como nem tudo são flores, o ingress deve ser configurado com um endereço externo para conseguirmos acessar.<br>Por sorte, estamos usando o Minikube e existe uma funcionalidade de tunelamento, onde atribuiremos os IP&#39;s externos (No caso de uma aplicação real configurada em serviços como EKS ou GKE, o ip externo será setado por algumas outras configurações. Esse procedimento serve apenas para testes).</p><p>Para ativarmos o tunelamento, podemos aplicar:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jzFMbxhJPUAl5vPkdgjGJQ.png" /></figure><p>Observe:</p><ul><li>O ingress-nginx-controller está sendo tunnelado pelo minikube, o que nos dará um ip externo. Isso é perfeito para o caso que precisamos. O ingress que configuramos préviamente será acessado por esse controller e redirecionará ao nosso deployment.</li></ul><p>Portanto, podemos pegar o endereço do serviço ingress-nginx e testarmos entrar na nossa aplicação!</p><p>Basta apenas, no seu terminal, digitar:</p><pre>kubectl get svc -n ingress-nginx</pre><p>Assim:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oWUw4DeNV9hzQmmhTzkBYA.png" /></figure><p>Observe:</p><ul><li>No service “ingress-nginx-controller” temos o EXTERNAL_IP!</li></ul><p>Se acessarmos o endereço externo:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iU_o3Rbor24P4KYY_tpsgA.png" /></figure><p>Voilá! Acessamos a nossa aplicação via ingress, redirecionando pela service!</p><p>Esse proxy já faz o balanceamento de carga automaticamente e temos nossa aplicação funcionando perfeitamente!</p><p>Mas podemos melhorar mais ainda… E quando queremos alterar nossa aplicação? E se quisermos subir um novo frontend e validarmos uma parte dos acessos a ela? Assim seguimos com o ArgoRollouts para o Canary Deploy!</p><h4>ArgoRollouts e Canary Deploy!</h4><p>Agora que temos nossa estrutura configurada com o Deployment rodando, Services e Ingress funcionando, podemos seguir com os próximos passos.</p><p>Antes de mais nada, quero reforçar que escolhi o Canary Deployment como exemplo, mas da mesma forma podemos aplicar o BlueGreen com apenas algumas alterações.</p><p>A teoria envolvida foi explicada no início do artigo. Caso não tenha lido, é só rolar para cima e depois voltar aqui.</p><p>Para mérito de arquitetura, o Argo funciona como uma controller. Assim como mencionamos anteriormente, é um serviço que monitora os namespaces e adequa-os para funcionarmos como desejamos (se assim podemos dizer rs). A imagem que o próprio Argo disponibiliza para representar mostra bastante sobre como ele controla os PODs e as estratégias escolhidas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*801T1ujDHA5lEhED2op3yA.png" /><figcaption>Got from <a href="https://argo-rollouts.readthedocs.io/en/stable/architecture/">https://argo-rollouts.readthedocs.io/en/stable/architecture/</a></figcaption></figure><p>Para começarmos, a únicas alterações que temos que fazer no nosso deployment são as seguintes:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/35f3871f5adf952b98b6cd75ff3da381/href">https://medium.com/media/35f3871f5adf952b98b6cd75ff3da381/href</a></iframe><p>Perceba:</p><ul><li>Trocamos a APIVersion para: argoproj.io/v1alpha1, assim mantemos o rollout definido pelo Argo</li><li>Trocamos o Kind para “Rollout”</li><li>Adicionamos o campo strategy! Nele definimos a estratégia que queremos, junto às especificações! Dessa forma controlamos o Canary Deploy!</li></ul><p>Antes de aplicarmos as alterações que fizemos em nosso deployment, devemos fazer a instalação do Argo.</p><p>Para méritos de comparação, o argo não é nada mais que uma Controller. Assim como instânciamos o ingress-nginx, devemos instanciar o como controller.</p><p>Você pode seguir o passo a passo de instalação do guia oficial (recomendo a instalação manual):</p><p><a href="https://argo-rollouts.readthedocs.io/en/stable/installation/">https://argo-rollouts.readthedocs.io/en/stable/installation/</a></p><p>Lembre-se, a instalação do Argo e a do plugin do Kubectl deve ser feita!</p><p>Garanta que a instalação deu certo, rodando:</p><pre>kubectl argo rollouts version</pre><p>Agora, antes de subirmos novamente nossos pods, podemos rodar:</p><pre>kubectl argo rollouts dashboard</pre><p>E encontraremos a aplicação de monitoramento dos Rollouts rodando localmente!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ww4zj1MMEZnwJcsf_g6FMA.png" /></figure><p>Assim, podemos deletar o nosso Deployment com:</p><pre>kubectl delete deployment angular-canary</pre><p>E subir a nossa nova instância do Rollout!:</p><pre>kubectl apply -f k8s/rollout.ts</pre><p>E nossa aplicação logo aparecerá!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5JYw8SOkOQoM7S8axE0JvQ.png" /></figure><p>Ao abrir o Rollout, logo veremos algumas coisas interessantes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*81DIL0hODOmSfU7-9kT3aw.png" /></figure><p>Perceba:</p><ul><li>Os steps são os passos que definimos no arquivo de rollouts!</li><li>A estratégia que configuramos foi a Canary!</li><li>A imagem que está rodando aparece nos containers e a versão estável é especificada.</li></ul><p>Bom, agora que temos o nosso rollout configurado, podemos brincar um pouco mais.</p><p>Lembra da label que deifnimos na V2: “THIS IS THE SECOND VERSION”?</p><p>Bom, ela está rodando atualmente. Se abrirmos o endereço externo que definimos com o tunelamento do controller ingress-nginx, veremos ela.</p><p>Agora, mudando a versão da imagem no nosso rollout para a V9 que contém a segunda label que definimos: “THIS IS THE OTHER VERSION HEEEERE”, veremos o canary em progresso.</p><p>Dessa forma, no rollout:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IDGtXNiLS7lFiAz8hpQGPQ.png" /></figure><p>E aplicamos o rollout, novamente, com:</p><pre>kubectl applt -f k8s/rollout.yml</pre><p>Veremos que o ArgoRollout aplicou a primeira versão em um único pod! (correspondente a 20% — que foi o que definimos nas strategies do próprio Rollout).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cgKq3Xndea37pLrP2sekMQ.png" /></figure><p>Como o nosso serviço usa um LoadBalancer e redireciona automaticamente de acordo com a disponibilidade dos PODs, se acessarmos o nosso endereço externo do ingress e dermos Ctrl+F5 (atualizar a página, removendo o cache), veremos que em algum momento iremos cair na nova aplicação!</p><p>Porém, se você fez isso,<strong> perceberá que vai dar alguns erros</strong>. Muitas vezes sua aplicação não irá carregar.</p><p><strong>Muito provavelmente você terá uma tela em branco, sem nenhuma resposta do navegador.</strong></p><p>Se inspecionar o motivo, verá que a request tentou acessar um recurso do POD, mas ele não existe!</p><p>Quando usamos o Ingress para acessar o POD, ele redireciona ao service que, por sua vez, redireciona a aplicação. O problema é que o Ingress não tem distinção de sessões! O que está acontecendo é o seguinte:</p><ul><li>Você tenta acessar o recurso</li><li>O ingress te redireciona corretamente para a aplicação</li><li>Assim que você estabelece uma comunicação com o POD, o browser faz o fetch para buscar os recursos estáticos</li><li>Cada fetch para buscar recursos estáticos é uma nova request!</li></ul><p>Percebeu o problema?<br>O que acontece é que cada request que busca estáticos está sendo redirecionada para um POD diferente!</p><p>Isso certamente irá trazer problemas no futuro, dado que estamos apenas manipulando 2 aplicações que se parecem muito!</p><p>E se arquivos estáticos que nunca existiram em um POD forem requisitados? Entende?</p><p>Para resolvermos esse problema, temos que adicionar ao Ingress e ao Service uma afinidade.</p><p><a href="https://buddhimau.medium.com/affinity-in-nginx-ingress-controller-1-247e733229c5">Affinity In Nginx Ingress Controller-1</a></p><p>Basicamente vamos falar ao ingress: “Preciso que você mantenha a sessão desse cara por pelo menos X tempo e redirecione ele sempre para a mesma aplicação. Assim eu consigo garantir que o usuário vai estar acessando o mesmo POD que ele acessou da primeira vez”.</p><p>Se não ficou claro o problema, imagine que o cliente acesse um POD que tem uma nova feature. Em seguida ele fecha o navegador e acessa novamente e a feature não existe nesse outro POD que foi redirecionado. Pronto, uma confusão na cabeça do usuário.</p><blockquote>Lembrando que o que vamos aplicacar aqui é monitorado pelos cookies.<br>Existem outros métodos com istio, por exemplo, que conseguem fazer essa afinidade com mais eficácia. Porém, no momento, não é o objetivo aqui.</blockquote><p>Portanto, para adicionarmos essa afinidade, na nossa Ingress adicionaremos:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1d736db9032a7048851a312421366e4/href">https://medium.com/media/b1d736db9032a7048851a312421366e4/href</a></iframe><p>Perceba:</p><ul><li>Nas annotations adicionamos: /affinity: “cookie”</li><li>Também temos o session-cookie-expires. Esse parâmetro é controlado por segundos! Me dei o luxo de colocar apenas “1” segundo para testes que vamos fazer por aqui.</li></ul><p>Já na nossa service, faremos:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4d28391a83c4406aed6888f1b91435c5/href">https://medium.com/media/4d28391a83c4406aed6888f1b91435c5/href</a></iframe><p>Perceba:</p><ul><li>Adicionamos a sessionAffinity apontando para o IP do cliente</li><li>Adicionamos a configuração de apenas 1 segundo de timeout (APENAS PARA TESTES)</li></ul><p>Dessa forma, subindo o nosso serviço e ingress novamente, veremos que não teremos mais erros em relação a renderização das versões da nossa aplicação!</p><p>Um teste a ser feito é: abrir o google Chrome em uma aba privada (não permite os cookies) e começar a atualizar a página. Dessa forma, veremos que eventualmente haverá uma troca entre as aplicações.</p><p>Lembre-se que a nossa carga está em 20% na nova versão e 80% na versão anterior. Qualquer coisa consulte no dashboard do argo para ver o estado dos seus PODs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aq-fvPUG7wnjdifZbtVUzg.gif" /><figcaption>Atualizações sequenciais pelo ip externo do ingress redirecionando para 2 aplicações diferentes.</figcaption></figure><p>No mais, lembre-se SEMPRE de adicionar mais tempo à configuração de afinidade.</p><p>Para seguir com o Rollout ou prosseguimos pelo terminal ou pelo dashboard do Argo.</p><p>Dessa forma:</p><pre>kubectl argo rollouts promove &lt;ROLLOUT_NAME&gt;</pre><p>ou:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jLTuXRk9IEvQRV6ixyItNw.png" /></figure><blockquote>Para outros commandos via CLI, rodar no terminal kubectl argo rollouts -h</blockquote><h3>Conclusão</h3><p>No mais, acredito ser isso. Não só por falar de frontend, mas acredito que estratégias de controle de clusters devem ser aprendidas por todo e qualquer desenvolvedor.</p><p>Todos irão fazer algum deploy um dia.</p><p>O repositório base dessa aplicação que usei se encontra aqui:</p><p><a href="https://github.com/victorfjansen/angular-canary-deployment2">GitHub - victorfjansen/angular-canary-deployment2</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97be2b706a0f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Observables — deep in dive]]></title>
            <link>https://medium.com/@victorfjansen/observables-deep-in-dive-4b5953689960?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/4b5953689960</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[angular]]></category>
            <category><![CDATA[rxjs]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Sun, 04 Jun 2023 13:43:15 GMT</pubDate>
            <atom:updated>2023-09-28T17:40:13.549Z</atom:updated>
            <content:encoded><![CDATA[<h3>Observables — deep in dive</h3><p>Certainly, if you’ve used Angular or at least the Observer standard, you know what I’m going to talk about here. Many people use observables knowing their concepts, but don’t know what they actually use under the hood.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OpaHO4s_OKpYX7-4Uv2ymQ.png" /></figure><blockquote>Before we start, in this article I’m assuming that you already know about the pattern and how we use this architecture on a daily basis. I’m also assuming that you know how to manipulate classes, observables and that you understand JS advancedTypes.</blockquote><h4>TL; DR</h4><p>Famous for implementing a push-based architecture, Observables, which comes from the RxJs lib, has an amazing internal mechanism. And not just because of its functionalities, but because its internal architecture has been very well thought out from start to finish.</p><p>As much as we know how to use the aspects of observables, we are generally not used to understanding their internal processes. They often don’t know why an observable only works when the subscribe function is called or why memoryLeaks occur if we don’t unsubscribe.</p><p>After all, what causes memory leaks in this implementation? Do you know?</p><h4>A more in-depth view</h4><p>In principle, we know that Observables are classes that receive a callback in their constructor, where they can call the functions next, error, complete and so on…<br>In this way, we can implement:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*UZ_4V3WuPIAgI6kW8FT6Qg.png" /></figure><p>And then we can call the subscribe method of that class:</p><p>So:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*YWoWUorILZlPoPpgjmvcaQ.png" /></figure><p>Watch:</p><ul><li>The callback can access the Observable’s internal methods and execute functions that are reflected in the subscribers.</li></ul><p>But how does this work internally?</p><p>Let’s first look at the callback implemented.</p><p>If we pull up the Observables source code, we’ll see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*KxAaByw7fnu_3iwow39J5w.png" /></figure><p>Watch:</p><ul><li>In the constructor we get the subscribe . This subscribe is a callback that has 2 parameters, one of which is *this* and the other is *subscriber* .</li><li>The *this* parameter works like a “fake this”. It’s used to add context to the callback’s this object. For the sake of explanation, it is disregarded when compiling the typescript. It really only serves semantic purposes.</li><li>As our fakethis is disregarded in the compilation, we only have one parameter (which is the subscriber) and it is this that receives our callback that we put in the Observable call.</li><li>Ex: new Observable(**the callback goes here**)</li><li>It checks if this parameter exists in the constructor and assigns the value of this callback to a private property (_subscribe). This is where the magic happens (we’ll see later).</li></ul><blockquote>I’ll leave the reference to the fakethis issue here. If you want to know more about it, follow the official doc:</blockquote><p><a href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-0.html#specifying-the-type-of-this-for-functions">Documentation - TypeScript 2.0</a></p><h4>Observable.subscribe() — Creating the Observable!</h4><p>Well, now that we’ve seen how the callback typing works and that it’s stored in the _subscribe property (a very important point, don’t forget it), let’s see how the observable’s .subscribe() function works.</p><p>If we pull up the source code, we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*HzEFaN28z0I5y2U4qhk26g.png" /></figure><p>Watch:</p><ul><li>It receives an object (which is of type Partial&lt;Observer&lt;T&gt;&gt; ) or a callback or a null value.</li><li>Based on the parameter received, it creates a new Subscriber with this information.</li></ul><p>And we’re left wondering: What the hell is this new Subscriber() ?<br>The Subscriber class is nothing more than a class that manages the functions we pass as parameters to our .subscribe(**parameter**) function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DCLK2n_U-Q20LC2U31mMkA.png" /></figure><p>Check it out:</p><ul><li>We receive the destination in the constructor (which is the parameter we pass in our .subscribe(**parameter**) function).</li><li>In the destination instance, we create a safeObserver , where we pass the parameters we received from the .subscribe(**parameter**) function.</li><li>From there, we can see the new instance we’ve created — the safe Observer.</li></ul><p>In this case, we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3DnAlCZf80rdSC55rKZhiw.png" /></figure><p>Which in turn calls a new ConsumerObserver. Let’s see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*-CW8x_OZVWH1upUzJBpHHw.png" /></figure><p>Finally, our destination instance allocated to our Subscriber receives nothing more than a formatted object with the necessary error handling from our next, complete and error functions that we receive via a parameter when we call the .subscribe(**parameter**) function.</p><p>Returning to our Observable, after creating the Subscriber instance, we see that a .subscriber.add() function is called.</p><p>This way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sFWhYJcFoAqcGG9UBYaehw.png" /></figure><p>And again we ask ourselves. What happens in this .add function?</p><p><strong>This is where the observable registrations take place!</strong></p><p>Opening our *Subscriber* class we will see that it is extended to a new class called Subscription . This class is responsible for all observable subscriptions in our application (and also for unsubscribing haha). As we can see, the add:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*yU8inEy8_CO-_IC-xUrW5g.png" /></figure><p>Watch:</p><ul><li>First, it checks to see if there really is an entry coming in (if it’s valid — true) and if it’s different from the context itself (from this).</li><li>Then it checks if the subscription is already closed. In this case, if the Observable’s unsubscribe has already occurred (this is done by this.closed — this process occurs in the unsubscribe function).</li><li>If it isn’t closed and isn’t from the context itself, it still unsubscribes for safety and then creates it in the _finalizers instance, assigning a JS advancedType called Set().</li><li>Creating the finalizer, we assign the teardown parameter to this Set()!</li></ul><p>Got it? An inscription (which fundamentally generates memoryLeaks) is nothing more than a new Set() that receives the teardowns! It’s pretty simple after all!</p><p>And now we wonder: What on earth are these teardowns?<br>The teardown passed to the add() function of the Subscription class is nothing more than <strong>our callback that we passed in the Observable constructor!</strong></p><p>Pulling up our .subscribe(**parameter**) function again, we have:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e5Xh43EaDhGhooBwhJJnNg.png" /></figure><p>This time, presenting it with the built-in _trySubscribe() function.</p><p>Remember at the beginning, where I told you not to forget that we assign the observable callback to an instance called _subscribe ? So let’s use it!</p><p><strong>In point 2:</strong></p><p>Note that, if all goes well, we return this callback with our subscriber. This is where our function (the one we put as a parameter of .subscribe(**parameter) ) is executed.</p><p>To make it clearer:</p><p>Suppose I created an Observable with the following callback:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*jzTbMZG97VAgIk7TUUCVfQ.png" /></figure><p>When we replace this param value with our Subscriber instance (and remember that although Subscriber is a class, it is also an object), the next function called is the one we put as a parameter of the .subscribe(**parameter**) function.</p><p>I didn’t show it, but pulling up our Subscriber class there’s a method called next()!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*Y2n35_IPE-ngbun84V8MSA.png" /></figure><p>Which in turn calls destination.next() (The instance we assigned when we created Subscriber() — as ConsumerObserver):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*1H74Ohok888OyjeM-BjWOw.png" /></figure><p>All this to simply call the same function we declared in Subscribe! Confusing, isn’t it? hahaha</p><p><strong>In point 1:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e5Xh43EaDhGhooBwhJJnNg.png" /></figure><p>Still looking at our subscribe() function, after triggering the call of the function we passed as a parameter, we still add the return of this object as our teardown, in other words, we add it as our subscribe in that Set().</p><p>Finally, we return the Subscriber itself, being able to access the add , remove , closed , unsubscribe methods. Everything we’ve already seen here and how they work internally.</p><h4>Finally, the unsubscribe</h4><p>Unsubscribe is nothing more than the manipulation of Set() defined by the Subscription class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*CfdzHtLx4IDzc5azfpGBCw.png" /></figure><p>Look:</p><ul><li>We can receive this initialTeardown via constructor (which is not our case — other implementations of this class take this approach)</li><li>We check if the closed property is false and then assign true to it (when it is true, it means that the inscription is closed).</li><li>Checks if the property we received via injection is a function. If it is, it tries to execute the unsubscription</li><li>Apart from that, we take the existing _finalizers referring to the this object and do the unsubscribe process (see that we assign the _finalizers the value of null. Not only have we unsubscribed, but we’ve also cleared the Set() reference that was previously set).</li><li>Then, if there are errors, we throw them into the errors array and deal with them.</li><li>The execFinalizer function is for when the subscription is an observable. Thus, it calls finalizer.unsubscribe() . See:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*82KsB1BndiQsyMsx3wW-9A.png" /></figure><h4>Additional: Pipe TakeUntil 🤯</h4><p>I bet you’ve wondered how things work (and so have I)!<br>That’s why I thought it would be worth bringing you how the takeUntil pipe works and how it manages unsubscriptions.</p><p>Like every custom pipe we can create, we will have a function that returns an Observable. Let’s see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RJBGPNfoWltX48pdYs-l7w.png" /><figcaption>Reference 1</figcaption></figure><p>Look:</p><ul><li>We receive the notifier (classic unsubscribe$ ) as a parameter of the function</li><li>The source parameter that we receive in the callback is our original observable (which we are placing the takeUntil pipe in).</li><li>In this callback, we create a new Observable, where inside it we create a new observable with the from operator</li><li>In this from operator, we pass our notifier (our unsubscribe$ ), we subscribe and, inside the subscription, we pass the operate() function with the very observable we created in the callback, completing itself — destination.complete()</li><li>Then we check that the destination is complete (destination.closed) and subscribe to our original observable, passing our destination (which is complete).</li></ul><p>Let’s break it down:</p><p>Remember that the from operator creates a new observable from arrays.<br>When we subscribe to the observable in the from operator, we pass it a function called operate() . But what is this operate?</p><p>It is a function exported from the Subscriber class that returns a new instance of the Subscriber itself!</p><p>Let’s see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*FasUJaPLEdoGk-vQw76puQ.png" /><figcaption>Reference 2</figcaption></figure><p>Notice that inside this operate function (in reference image 1) we pass the destination (observable that we created inside the callback). In other words, we are creating a Subscriber from the callback of another observable.</p><p>However, still in this operate function, we pass destination.complete() and the complete function as noop .</p><p>This noop is simply an empty function, which will overwrite the complete function of the Subscriber instance created!</p><p>In this case, it’s important to remember the assignment processes that take place in the Subscriber class constructor. Let’s see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ThxFyj1UzfmhLIFP4fr2BA.png" /></figure><p>Look:</p><ul><li>It checks if we have passed the overrides in the constructor (which is our case with the operate function)</li><li>Define the destination (just as we saw before)</li><li>Assigns to _completeOverride the function of the overrides object (in this case, it’s our noop )</li><li>Just below, on the last lines, we have the following assignment:<br>*this._complete = this._completeOverride ? overrideComplete : this._complete*</li><li>Then the _complete function is empty! and there’s no point in calling it any more! The unsubscribe will not be performed!</li></ul><p>But what now? Do we stop unsubscribing from observables? No!</p><p>Going back to reference image 1, in the last line, we still have the following expression:</p><p>!destination.closed &amp;&amp; source.subscribe(destination);</p><p>Look:</p><ul><li>We check that the destination’s observable was complete (in this case it wasn’t, because we replaced the complete function with an empty one — this in the new Subscriber we created, not in the original destination).</li><li>With a value of false in closed, the expression becomes true because of the negation</li><li>And we signed up by passing on the destination.</li></ul><p>and now?</p><p>Now, in our component where we implemented the takeUntil pipe, in our Subject that we created (unsubscribe$ ), we give the classic unsubscribe$.next() and unsubscribe$.complete().</p><p>Thus, we finalize the inscription in our original observable, in the Subject that we created in the context of the component, in the observable of the from operator (after all, there is no longer a notifier) and we clear the references in memory through the ngOnDestroy() cycle.</p><h4>Finalization</h4><p>I thought about commenting on Subjects and BehaviorSubjects as well, but that’s for a later post.</p><p>They’re the same implementation, but with a few additions 👀</p><p>All in all, I recommend opening the original RxJS code to explore the topic further and see other types of implementation.</p><p><a href="https://github.dev/ReactiveX/rxjs">https://github.dev/ReactiveX/rxjs</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b5953689960" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ForRoot Pattern — Inicialização e gestão de dependências dentro de aplicações]]></title>
            <link>https://medium.com/@victorfjansen/forroot-pattern-inicializa%C3%A7%C3%A3o-e-gest%C3%A3o-de-depend%C3%AAncias-dentro-de-aplica%C3%A7%C3%B5es-dd82f2c1aa9?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd82f2c1aa9</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[angular]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Mon, 22 May 2023 03:49:58 GMT</pubDate>
            <atom:updated>2023-05-22T03:49:58.336Z</atom:updated>
            <content:encoded><![CDATA[<h3>ForRoot Pattern — Inicialização e gestão de dependências dentro de aplicações</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mPT3RguvKX0AvTAPmjOXpQ.png" /></figure><p>Como gerir inicializações na aplicação? Como definir ciclos e processos para ocorrerem sem que precise de interferir nos fluxos dos templates? Como administrar informações de integrações externas pra manterem um SST e não virar um emaranhado de informações perdidas? São algumas das perguntas que vão ser respondidas aqui.</p><p>Em alguns dos sistemas mais completos que temos, sempre há integrações com outros softwares afim de complementar o objetivo do mesmo software. Me diz, com certeza já viu aquelas ferramentas de rastreamento como o GoogleAnalytics ou Mixpanel e teve que integrá-las ao seu projeto, mas ficou sem saber como fazer isso de início, certo?</p><p>Então, trouxe uma alternativa que não imprime complicações no seu projeto.</p><p>Se já utilizou o Angular algumas vezes, tenho certeza que se lembra do RouterModule.forRoot() . É sobre a mágica que acontece por debaixo dos panos que vou falar.</p><p>Antes de começarmos, quero deixar claro que estou partindo do princípio que estamos olhando o angular como uma plataforma de aplicações e não como um único sistema. Assim, vou construir uma integração visando vários projetos. Pra saber mais sobre:</p><p><a href="https://medium.com/disney-streaming/combining-multiple-angular-applications-into-a-single-one-e87d530d6527">Combining Multiple Angular Applications into a Single One</a></p><h4>Iniciando o Projeto</h4><p>Antes de começar, vou demonstrar uma integração com o Mixpanel (uma ferramenta parecida com o GoogleAnalytics). Lembrando que essa integração não se limita somente a essa implementação, mas que te oferece inúmeras possibilidades de aplicação.</p><p>Pra começar, peguei um projeto antigo meu que se trata de uma página onde você pode ver vários dogs, adicioná-los aos favoritos e coisa do tipo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0Fb66ScIp39Cn8WLA5n1WQ.png" /></figure><p>Uma página simples, porém que terá a integração de uma api nela.</p><p>O link do repositório:</p><p><a href="https://github.com/victorfjansen/DogsGallery">GitHub - victorfjansen/DogsGallery</a></p><p>Pra fazer a integração com o Mixpanel, é necessário criar uma conta na plataforma deles e pegar um token específico pra ter seu projeto configurado.</p><p>Caso queria seguir o passo a passo, fique a vontade pra continuar.</p><p>No mais, como vamos manter o mixpanel, eles já tem uma lib disponível e vamos utilizar ela pra facilitar a nossa integração.</p><p>Então no terminal colocamos: npm i mixpanel @types/mixpanel-browser</p><h4>Utilizando o ForRoot</h4><p>A ideia do forRoot é configurar o ambiente necessário (através de um módulo) com uma função estática que retorna o próprio módulo.</p><p>No caso da nossa integração, ficaria mais ou menos assim:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/0*IBHAMPJ8DA6xU0tn.png" /><figcaption>mixpanel &gt; src &gt; ng-mixpanel.module.ts</figcaption></figure><p>Observe:</p><ul><li>Configuramos um módulo que tem as suas propriedades normais (exports, declarations, imports, etc..)</li><li>O nosso módulo contém uma função estática chamada forRoot que retorna o tipo ModuleWithProviders&lt;NgMixpanelModule&gt; , sendo o ModuleWithProviders uma implementação genérica de:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/0*UnJFQFS3hRBewzi7.png" /></figure><ul><li>O método forRoot devolve nosso módulo, como se fosse uma integração normal.</li></ul><p>A mágica real acontece no array de providers, onde conseguimos configurar eventos pra acontecerem ao inicializar, ou definir Tokens de acesso dentro da nossa aplicação (que se referem à nossa implementação) sem sujar o template em outras partes da aplicação.</p><p>Segundo a documentação do Mixpanel, precisamos do projectToken , userId (usado pra identificação primária do usuário), e do debugMode.</p><p>Ou seja, são dados de entrada imutáveis dentro do projeto e que, para evitar possíveis conflitos, devemos definir um SST (single source of truth).</p><p>E como fazemos isso então? por meio de InjectionTokens. Vamos criar um novo injectionToken pra definir essas propriedades. Observe:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/0*Atas4YZC4ny70voy.png" /><figcaption>mixpanel &gt; src &gt; tokens &gt; ng-mixpanel-settings.token.ts</figcaption></figure><ul><li>Definimos um novo InjectionToken com uma factory, onde retornamos as propriedades que citamos acima.</li><li>A interface MixpanelSettingsViewModel apenas exige essas propriedades</li></ul><p>Pronto, agora retornando ao nosso módulo:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/0*68-9LtfdqVo7kSxX.png" /></figure><p>Observe:</p><ul><li>Agora podemos definir os valores do nosso InjectionToken a partir dos valores que o usuário colocar na definição do módulo. Logo, na nossa aplicação, sempre que fizermos uso do NG_MIXPANEL_SETTINGS_TOKEN , teremos os valores primários setados.</li><li>Tipamos o valor do nosso provider com a interface que criamos</li></ul><h4>Usando initializers</h4><p>Agora que temos o que precisamos, podemos começar a configurar as nossas dependências por meio de initializers.</p><p>Initializers não são nada a mais do que Providers que se aproveitam de outro InjectionToken do Angular chamado APP_BOOTSTRAP_LISTENER .</p><p>Pra mérito de explicação, o APP_BOOTSTAP_LISTENER é usado no ciclo de inicialização do angular, onde é visto como um array com factory functions para execução.</p><p>Se pegarmos o código fonte do angular, vemos uma função chamada _loadComponent que faz proveito da instância do Injector.get pra pegar os providers e executá-los.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*9J7_4VO7ZlMFAwjz.png" /></figure><p>Observe:</p><ul><li>Ele requer o uso de cada instância do InjectionToken com a propriedade multi pra dar seguimento à inicialização a aplicação.</li><li>Conseguimos observar que ele executa cada factory function a partir da referência existente da aplicação (isso no forEach).</li></ul><p>Agora que entendemos como esse injectionToken funciona, podemos voltar a nossa aplicação e configurar uma instância do mesmo para executarmos as ações necessárias antes da nossa aplicação inicializar.</p><p>Observe:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*Lot8_l-CBpBwWEiq.png" /><figcaption>mixpanel &gt; src &gt; initializers &gt; ng-mixpanel-initializer.initializer.ts</figcaption></figure><ul><li>Criamos um objeto e exportamos, tipando ele como um Provider</li><li>Definimos nossa propriedade multi como true (assim como vimos acima que é necessário)</li><li>Usamos uma factory function para definirmos uma ação do nosso ciclo de inicialização</li><li>Verificamos cada propriedade que precisaremos (para evitar erros futuros) e chamamos a função mixpanel.init que é requerida na documentação do mixpanel (definir o token do projeto e tudo mais).</li><li>Como quero ter garantia que esse inicializador está rodando em um Browser, defini o DOCUMENT , que vem direto do angular/core, como obrigatório. Não é necessário, mas evita complicações futuras da minha implementação.</li></ul><p>Agora que definimos um initializer, podemos alterar o nosso módulo da aplicação!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/0*1fTyu-llVisFWLMW.png" /></figure><h4>Usando nossas instâncias configuradas</h4><p>Agora que temos nossa aplicação configurada, posso criar um serviço que facilite a implementação dessas configurações.</p><p>Observe:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/886/1*VPImH5AKDVIA2CxUYPzpbw.png" /><figcaption>mixpanel &gt; src &gt; services &gt; mixpanel-track.service.ts</figcaption></figure><ul><li>Agora consigo criar uma função track, onde posso usar em qualquer lugar/função que ativa o evento de track existente no mixpanel</li><li>Consigo usar também as configurações de quando inicializei o módulo. Nesse caso, obrigo toda requisição a possuir o ID do usuário</li></ul><h4>Pra finalizarmos</h4><p>Agora que implementamos as nossas configurações, a versão final do módulo seria essa:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*tC6fFX1T1-GQIJhP.png" /></figure><p>Observe:</p><ul><li>Coloquei o nosso serviço nos providers, assim não preciso definir ele como inRoot</li><li>defini uma função estática chamada forChild onde retorna o módulo. Essa vai apenas pra garantir que não exista nenhum módulo cego em relação às nossas implementações (observe que adicionei diretivas no declarations e no exports)</li><li>Adicionei uma diretiva que faz a implementação do nosso serviço de track. Assim podemos evitar ficar injetando o nosso serviço em todo lugar.</li></ul><p>Como ficou a diretiva:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/0*Kuy_B08KC5Z1kQjQ.png" /></figure><p>Como ficou nosso appModule:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/0*PvjAFxczEhlPHm0n.png" /></figure><p>Até mesmo recomendo que coloque algumas dessas variáveis nas environments da aplicação. Assim conseguimos garantir a visão de plataforma do angular e evitar descentralização de regras de negócio do código.</p><p>No mais, mediante a nossa integração, fiz uma implementação da nossa diretiva e adivinhem?</p><p>A nossa integração com o mixpanel ocorreu como o esperado:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*OQbvAwGn9IIQDZFm.png" /><figcaption>ultima requisição feita ( a primeira de baixo pra cima )</figcaption></figure><p>Acredito que era isso que queria demonstrar nesse artigo, falando sobre tokens de inicialização e sobre o ForRoot pattern.</p><p>Caso queira brincar um pouco com o projeto:</p><ul><li><a href="https://stackblitz.com/github/victorfjansen/DogsGallery?embed=1&amp;file=README.md">Victorfjansen - Dogs Gallery - StackBlitz</a></li><li><a href="https://github.com/victorfjansen/DogsGallery">GitHub - victorfjansen/DogsGallery</a></li></ul><p>Alguns outros artigos que possa se interessar:</p><ul><li><a href="https://medium.com/@victorfjansen/resolution-modifiers-controlando-as-regras-de-inje%C3%A7%C3%B5es-de-depend%C3%AAncias-3b30f75735f2">Resolution Modifiers — Controlando as regras de injeções de dependências</a></li><li><a href="https://medium.com/@victorfjansen/providertoken-o-que-voc%C3%AA-mais-usa-e-n%C3%A3o-sabe-e97830ada19d">ProviderToken: O que você mais usa e não sabe</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd82f2c1aa9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Injection Scopes — modos de carregamento de serviços com NestJS]]></title>
            <link>https://medium.com/@victorfjansen/injection-scopes-modos-de-carregamento-de-servi%C3%A7os-com-nestjs-4c0bd1929689?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c0bd1929689</guid>
            <category><![CDATA[nestjs]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Wed, 15 Feb 2023 18:29:02 GMT</pubDate>
            <atom:updated>2023-02-15T18:31:25.734Z</atom:updated>
            <content:encoded><![CDATA[<h3>Injection Scopes — modos de carregamento de serviços com NestJS</h3><p>Assim como no Angular, no NestJS conseguimos ter serviços que são definidos nos providers e injetados diretamente nos construtores das diversas classes que podemos ter. Porém, o NestJS oferece algumas features interessantes para o carregamento de serviços.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HBMICJVxKSp1-ka6RyRUGQ.png" /></figure><h4>TL;DR</h4><p>Hoje em dia o NestJs pode ser usado pra várias finalidades. Tanto na construção de BFF’s quanto na construção de servidores monolíticos, orientados a microsserviços e serverless.</p><p>Independentemente da arquitetura que decidimos pras nossas aplicações, nunca temos uma bala de prata para todas as funcionalidades. Por isso quis trazer aqui algumas soluções, ou ao menos apresentar, de forma conceitual sobre o carregamento de serviços nas dependências internas do NestJS.</p><h4>Injection Scopes — Breve introdução</h4><p>Os injectable escopes se tratam, especificamente, de formas em que podemos carregar os nossos serviços na nossa aplicação. Sejam eles no padrão DEFAULT, TRANSIENT e REQUEST.</p><p>Mas antes de abordar cada uma dessas técnicas, acredito que seja necessário um conhecimento sobre como o NestJS faz o registro das dependências nas nossas aplicações.</p><p>Bom, e como funcionam afinal?</p><p>Toda módulo de uma aplicação contém as seguintes propriedades: Providers, Controllers, Imports e Exports.</p><p>Quando colocamos um serviço no array de providers, na verdade, estamos colocando um token que identifica o nosso serviço. É tudo questão de ser um sintax sugar para facilitar a vida do dev. Na real, estamos fazendo o seguinte:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*dx9ft5IF3yLUVg3wzlpgNw.png" /></figure><p>A questão é que geralmente o nome do token é o mesmo da classe, então podemos colocar sem abrir um objeto pra identificar o que é cada coisa.</p><p>Enfim, quando colocamos um provider, no ciclo de inicialização do módulo, o Nest lê o token, cria uma instância do serviço relacionado ao token e, toda vez que requisitamos o serviço no construtor de uma classe, ele devolve essa classe que ficou armazenada no cachê.</p><blockquote><em>Falo mais sobre isso em um artigo que escrevi dedicado ao registro de dependências no angular. Especifiquei de cabo a rabo como são usados os registros de deps e como funciona por debaixo dos panos. O processo é muito parecido com o do Nest. Recomendo ler antes de seguir. O link:</em></blockquote><p><a href="https://medium.com/@victorfjansen/providertoken-o-que-voc%C3%AA-mais-usa-e-n%C3%A3o-sabe-e97830ada19d">ProviderToken: O que você mais usa e não sabe</a></p><p>Essa é uma ótima ideia porque conseguimos controlar as instâncias que precisamos, seguindo um padrão singleton. Porém, vamos colocar um caso em cima da mesa:</p><p>E se tivermos um serviço enorme que realiza diversas operações pesadas no banco, além de processar muitos dados. O que fazemos? Já pensou que toda vez a nossa aplicação, em seu processo de bootstrap, vai ter que carregar e recarregar esse serviço?</p><p>Ou talvez, por alguma lógica que você aplicou, você precise de gerar uma nova instância desse serviço toda vez que ele for chamado, ignorando as outras instâncias desse mesmo serviço. O que você faz?</p><p>Vou te dar a resposta: seguir uma das opções abaixo!</p><p>Não vou abordar aqui o modo DEFAULT por conta de ser o padrão de todo serviço no nest, ok? Acho que todos sabem que é uma única instância gerada para toda a aplicação e que fica armazenada no cachê.</p><h4>TRANSIENT Scope</h4><p>O transient scope, a princípio, armazena o serviço também no cachê igual ao modo default. Porém, toda vez que uma instância é requisitada, uma nova instância da classe pertencente ao serviço é gerada.</p><p>Penso que um dos casos em que essa feature mostre importância, seja quando usamos esse serviço como uma espécie de factory, visando criar alguma nova funcionalidade.</p><p>Bom, e como ativamos? Simples. Se liga:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*VA19EcrDZtwN8bL7EhN56Q.png" /><figcaption>Jeito 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*C4P6cF9XQLYCWZw6FPbQ-A.png" /><figcaption>Jeito 2</figcaption></figure><p>As duas maneiras funcionam perfeitamente. Qualquer uma que você escolha será ideal.</p><h4>REQUEST scope</h4><p>O request scope foge totalmente dos padrões. Além de não ser armazenado em cachê nenhum, ele gera uma nova instância toda vez que uma Http request é feita. Ou seja, é mais isolado ainda.</p><p>Diferente do Transient scope (que uma nova instância é gerada automaticamente pra cada classe que o contém em seu construtor), ele só gera uma nova instância se um método interno da classe for chamado (httpBased).</p><p>E ainda, após ser utilizado, o processo de garbage collector é feito e a memória é liberada.</p><p>Pra ativar, é no mesmo esquema:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*RfdFWHWe97bLLnOjV0uU1w.png" /><figcaption>Jeito 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*b8Fx23hr3jLTGVUVuVLe2w.png" /><figcaption>Jeito 2</figcaption></figure><p>Por mais que pareça ser um grande benefício, o REQUEST scope tem uma perda de performance por ter que gerar uma instância do zero sempre que uma request for feita (além de alocar um espaço novo na memória, força o processo de garbage collector).</p><p>Na doc oficial do NestJS, por mais que tenham implementado, não recomendam usar esse método a não ser que você saiba o que está fazendo (na verdade, recomendam usar o padrão default pra tudo que puder).</p><p>Um dos casos recomendados para ser usado é quando você não está controlando a cadeia de chamadas da aplicação (quando uma lib, por exemplo, restringe a assinatura de uma função).</p><p>Existe um outro artigo falando sobre o porquê você deveria ficar longe de request scopesna maioria dos casos e acho importante ter o conhecimento dele. Caso queira conferir:</p><p><a href="https://guxi.me/posts/why-you-should-avoid-using-request-scope-injection-in-nest-js/">Why You Should Avoid Using Request-scoped Injection in NestJS</a></p><h4>Conclusão</h4><p>Bom, dependendo de como vai funcionar a arquitetura da sua aplicação, vale a pena alterar o scope do seu serviço. No mais, recomendo fortemente continuar usando o default scope .</p><p>Enfim, no mais, tenho alguns outros artigos escritos sobre NestJS, falando sobre segurança em aplicações, além de como montar e documentá-las.</p><p>Segue os links:</p><ul><li><a href="https://medium.com/@victorfjansen/class-validators-adicionando-seguran%C3%A7a-%C3%A0-sua-aplica%C3%A7%C3%A3o-nestjs-a4d708f884b5">Class-validators — Adicionando segurança à sua aplicação NestJS</a></li><li><a href="https://medium.com/@victorfjansen/bffs-pra-tratativa-de-dados-com-nestjs-dbfde05bdce8">BFF’s pra tratativa de dados com NestJS</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c0bd1929689" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Resolution Modifiers — Controlando as regras de injeções de dependências]]></title>
            <link>https://medium.com/@victorfjansen/resolution-modifiers-controlando-as-regras-de-inje%C3%A7%C3%B5es-de-depend%C3%AAncias-3b30f75735f2?source=rss-6d568ceabee5------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b30f75735f2</guid>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[angular]]></category>
            <dc:creator><![CDATA[Victorfjansen]]></dc:creator>
            <pubDate>Tue, 14 Feb 2023 17:16:45 GMT</pubDate>
            <atom:updated>2023-02-14T18:40:14.737Z</atom:updated>
            <content:encoded><![CDATA[<h3>Resolution Modifiers — Controlando as regras de injeções de dependências</h3><p>Você sabia que as resoluções de dependências dentro do Angular seguem uma regra hierárquica entre módulos? E sabe como conseguimos controlar a visibilidade de dependências para os componentes através dessa hierarquia? Fica ligado então!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_DcwNFCDWNTV8NkvPgifA.png" /></figure><h4>TL;DR</h4><p>Tudo nessa vida segue uma ordem, as dependency injection não poderiam ser diferentes! A questão é que conseguimos controlar tanto a ordem das injeções de dependências quanto a <strong>visibilidade </strong>delas dentro dos componentes e módulos.</p><p>Já pensou que, além de poder controlar renders, registros de dependências, você ainda pode controlar a visibilidade de dependências em componentes? Praticamente você tem o mundo na sua mão com isso!</p><blockquote><strong><em>Vou abordar o processo desde o início, explicando como funcionam as hierarquias dentro de um projeto Angular e como são as regras de resolução de dependências. Somente assim vamos conseguir entender a visibilidade de dependências em uma aplicação.<br>Não se preocupe se ficar um pouco desconexo de início, vou organizar as ideias ao longo do artigo.</em></strong></blockquote><blockquote><em>Só pra constar, pra entender todo o processo que vou escrever por aqui, recomendo que leia um artigo que escrevi sobre ProviderTokens por já ter explicado como os registros de dependências funcionam em aplicações Angular. Confira aqui:</em></blockquote><p><a href="https://medium.com/@victorfjansen/providertoken-o-que-voc%C3%AA-mais-usa-e-n%C3%A3o-sabe-e97830ada19d">ProviderToken: O que você mais usa e não sabe</a></p><h4>Hierarchical Injectors — O que são e onde vivem?</h4><p>Os hierarchical injectors não são nada mais que injectors. Todos são como containers que armazenam as instâncias de seus providers, porém cada um tem seu nível de prioridade.</p><blockquote>Não sabe o que é um injector? Vai ler o artigo que deixei em referência aqui em cima.</blockquote><blockquote>No mais, pra ficar mais visual, isso é um injector :</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/544/1*os3xtzKSdYB64QbajPsQCw.png" /></figure><p>Temos Injectors do nível mais alto ao mais baixo. Sempre que em uma aplicação ocorre o processo de bootstrap, o angular procura o<strong> nível mais baixo da aplicação que contenha um provider para instânciar um novo serviço.</strong></p><p>Sabe quando colocamos o array de provider dentro do nosso decorator @Component()? Então, estamos definindo, em um dos níveis mais baixos, o nosso array de dependências. Logo, por conta do angular sempre procurar dependências em níveis mais baixos primeiro, temos uma nova instância criada de um serviço.</p><p>Nessa brincadeira, temos 2 principais tipos de injetores hierárquicos nas aplicações, sendo eles:</p><ul><li>ModuleInjector — São os módulos e services configurados com os @Injectable() e @NgModule() .</li><li>ElementInjector — São os relacionados aos nossos componentes da DOM. Interessante comentar que todos os Elementos da DOM tem o ElementInjector, porém a maioria fica vazio por conta de não precisarmos de colocar providers neles (justamente o nosso caso do decorator @Component() ou até mesmo do @Directive() .</li></ul><h4>Modules Injector — Camadas de mais alto nível</h4><p>Primeiro vou abordar sobre as camadas mais altas dos injectors — os Modules Injectors. Escolhi abordar sobre eles primeiro por serem os que estamos acostumados a manipular (ou não haha).</p><p>Bom, até mesmo entre as camadas mais altas, temos uma hierarquia. Sendo elas: rootModuleInjector, ModuleInjector e NullInjector().</p><p>Não precisa se preocupar com decorar isso. Afinal, todos eles fazem a mesma coisa, mas em níveis diferentes. Todos registram dependências e as proveem aos módulos e componentes.</p><p>Pra ficar mais visual, se liga na hierarquia:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*Xzplj3Ht_YVLmLQNxam01Q.png" /><figcaption>Tenha em mente que o NullInjector é o de nível mais alto!</figcaption></figure><p>Sabe quando definimos em um serviço o providedIn: &quot;root&quot; ? Então, tratamos diretamente com o rootModuleInjector. Por conta disso, esse serviço pode ser acessado de todo lugar da aplicação sem restrições. É como se colocassemos esse serviço nos providers do nosso AppModule.module.ts, sabe?</p><p>Já o ModuleInjector é criado pelo processo de bootstrap do Angular (sendo uma camada superior). Ele é criado quando essa função aqui entra em vigor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*1vkxgPvNa6LDN92jbRr4OA.png" /></figure><p>E por curiosidade, qual a funcionalidade de um injetor superior ao rootModuleInjector? É justamente compartilhar informações de plataformas entre aplicações!</p><p>Estranho falar disso, mas temos que olhar pro angular não como um framework, mas sim como uma plataforma! Podemos ter mais de uma aplicação rodando no mesmo projeto! Por conta disso que existe essa camada superior a de root .</p><p>Pra variar um pouco, o time do Angular ainda colocou mais uma camada que é justamente o @NullInjector() (que é o nível mais alto de todos os injectors). Ela serve unicamente e exclusivamente para definir que realmente não existe nenhum registro de dependência na árvore, caso você esteja buscando um serviço que não tenha sido registrado em lugar nenhum.</p><p>Pense no seguinte cenário: Você criou o serviço X, mas não registrou em lugar nenhum. O angular, em seu processo de busca de dependências e inicialização de aplicação, busca desde o componente mais afastado do topo da árvore até o topo da árvore algum Injector que registre esse serviço X. Caso ele chegue ao nível do NullInjector (camada mais alta que pode existir em uma aplicação), o serviço não foi registrado em lugar nenhum.</p><blockquote><em>Infos adicionais:</em></blockquote><blockquote><em>- Caso você configure um provider no AppModule, ele vai sobrescrever os metadados definidos no @Injectable() (no caso, o providedIn: “root”)</em></blockquote><blockquote><em>- Usar o providedIn no @Injectable() é </em><strong><em>melhor </em></strong><em>do que configurar um provider em um @NgModule(). Isso, por conta de gerar um bundler menor devido ao processo de tree-shaking (caso queira ler sobre o tree-shaking do angular, </em><a href="https://angular.io/guide/architecture-services#providing-services"><em>clique aqui</em></a><em>)</em></blockquote><p><strong>ElementInjector — Camadas do nível mais baixo</strong></p><p>Como disse antes, o Angular cria um ElementInjector implícito pra cada elemento do DOM, porém eles ficam vazios até preenchermos com um provider ou viewProvider .</p><p>Pra ficar mais visual:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/355/1*rVyawE0o429EM2sGkKneUQ.png" /></figure><p>Nos componentes de baixo nível não temos nada demais. Mas é importante comentar que essas dependências que colocamos no array de providers <strong>ficam disponíveis</strong> aos componentes filhos e diretivas contidos nele.</p><h4>Tudo isso para chegarmos onde queríamos — Resolution Rules</h4><p>Beleza, quais são as regras de resolução afinal?</p><p>A resolução terá 2 fases principais:</p><ul><li>1 — Verificará primeiro os ElementInjetors</li><li>2 — Depois irá verificar os ModuleInjectors</li></ul><p>Depois, as outras regras:</p><ul><li>No geral, como foi falado antes, o Angular verifica o array de providers do ElementInjector, depois passa ao ElementInjector do componente Pai e assim em diante. Quando não encontra nada, passa aos ModuleInjectors e, se não acha nada de novo, lança um erro de compilação.</li><li>Caso ele encontre mais de um lugar declarando o mesmo InjectionToken, ele usará o primeiro que ele encontrar.</li></ul><h4>Resolutions Modifiers</h4><p>A cereja do bolo vem agora! Sabe todas essas regras que definimos e falamos que ocorrem no processo de resolução de dependências? Então, podemos alterá-las completamente! E tudo isso com os resolution modifiers.</p><p>Exemplos de manipulações:</p><ul><li>E se por algum motivo queremos adicionar uma dependência como opcional?</li><li>Se quisermos que ela retorne um null ao invés de travar nossa aplicação inteira?</li><li>E se quisermos parar uma procura de dependências no meio da inicialização da aplicação?</li></ul><p>No geral, temos 4 modificadores, sendo eles:</p><p><strong>@Optional() :</strong></p><p>Define a dependência que você injeta como opcional. Se o angular não encontrar nenhum provider pra uma dependência específica, ele retorna null . Geralmente é usado com os outros modifiers pra evitar erros.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*SoFGqwI9DvXdr9yl-h0Ecw.png" /></figure><p><strong>@Self():</strong></p><p>Usando o self, o angular vai olhar somente para o ElementInjector do atual componente ou diretiva. Por eliminar todo o processo de procura que o angular faz em outros ElementInjectors , pode acontecer de dar algum erro (não achar a dependência no atual ElementInjector ). Por isso, é comum associarmos ao uso do @Optional() .</p><p>Pra ficar mais visual:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*zRBUF4EN2drzkl2oA3XZ3w.png" /></figure><p>Observe:</p><ul><li>Nesse componente, ao tentar usar o someService , será retornado null . Isso por conta de não ter sido provido nenhuma dependência no array de providers e por termos definido que o angular vai olhar somente pra aquele ElementInjector .</li></ul><p>Já nesse caso conseguimos pegar e usar normalmente o nosso serviço:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/1*PQi3AwpIkwxS0Vf5-w1LPA.png" /></figure><p>@SkipSelf():</p><p>O SkipSelf é exatamente o oposto do Self . Ao invés de focar a leitura do injector do atual ElementInjector , ele passa a leitura pro componente Pai.</p><p>Então mesmo que você coloque um provider em um componente filho, será feita a leitura do componente de nível superior. E como anteriormente, também é usado com o Optional .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/693/1*7YVWesO7nkcRacBEiSyjhA.png" /></figure><p>Observe:</p><ul><li>Pra prevenir erros, colocamos o Optional .</li><li>Nesse caso, ele ignora totalmente o SomeService definido nos providers e passa pro componente pai. Não importa se é um ElementInjector ou um ModuleInjector .</li></ul><p><strong>@Host():</strong></p><p>O Host é quem define a última camada de procura de injectors. Se lembram que comentei que existe o NullInjector enquanto falava dos ModuleInjectors? Então, no caso geral, o NullInjector é a linha de parada, porém, com o Host , definimos a nossa nova linha de parada.</p><p>E é claro, novamente usamos o Optional pra evitarmos erros em momentos de compilação.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/1*oI3Fg2HdKxe8GXNLm6yCkg.png" /></figure><h4>E como usamos isso afinal?</h4><p>É difícil de achar uma aplicação pros resolutions modifiers nas aplicações do dia a dia, porém, tem alguns casos que se tornam no mínimo interessantes. No geral, podemos muito bem aplicar os resolutionModifiers em <strong>serviços</strong>!</p><p>Pensa o seguinte: Vamos supor que criemos um serviço X em que, a cada componente que injete ele, seja criada uma nova instância (colocamos nos providers do @Component() ). Beleza, agora nós queremos fazer uma manipulação somente com aquela instância para aquele determinado componente. Como fazemos? Justamente com o modifier @Self() :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/670/1*IH7JlPeyDLqAiYq6ycUm2g.png" /></figure><p>Desse jeito, podemos acessar a única instância do serviço gerado para o componente e modificar como quisermos. No caso, obviamente seria mais fácil acessar a instância do ManipulatedInstanceService dentro do componente, mas imagine: E se você criasse uma factory através desse esquema do @Self() decorator? São inúmeras possibilidades a serem exploradas.</p><p>Nisso, fora e dentro dos serviços podemos fazer manipulações com a regras de resolução de injeções.</p><h4>Adicional — View Providers</h4><p>Não cheguei a comentar por aqui, mas também existem os viewProviders . Basicamente tem os mesmos objetivos dos Providers, porém eles limitam a visibilidade de seus providers.</p><p>Bem, como assim?</p><p>Se lembram que comentei que o angular procura as dependências do nível mais baixo ao mais alto? Então, seguindo o processo que já descrevi por aqui, ele vai em cada ElementInjector e procura nos providers os ProviderTokens para poder gerar instâncias de suas classes. Mas e se quisermos isolar esse escopo de providers para funcionar unicamente com uma camada de componente? Nós usamos os viewProviders!</p><p>Pra ficar mais clara a ideia, observe o fluxograma:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*7cthaSZknIu7mSUJqCQ_Qw.png" /></figure><p>Observe:</p><ul><li>Declaramos o nosso provider no componente mais alto. Logo, todas as instâncias filhas teriam que pegar a referência do ProviderToken registrado nele.</li><li>No segundo componente (de cima pra baixo), interrompemos o registro de dependência, porém colocamos os viewProviders. Logo, o provider token vai assumir um novo valor, <strong>mas lembrando que isso só serve para esse componente.</strong></li><li>O terceiro componente vai usar o provider que registramos no <strong>primeiro</strong> componente. Mesmo que o segundo tenha alterado o fluxo de DI, ele alterou somente para ele.</li></ul><p>O próprio Angular deixa na documentação um exemplo no stackblitz para ser consultado. Caso queria analisar com mais cautela é só <a href="https://angular.io/generated/live-examples/providers-viewproviders/stackblitz.html">clicar aqui.</a></p><h4>Conclusão</h4><p>Acho que deu pra ter um bom parâmetro dos ResolutionModifiers e sobre como eles funcionam.</p><p>Agora é imaginar como podem ser usados na interrupção de resoluções de dependências pra gerar features interessantes. Acredito até mesmo que aquele exemplo do service com o @Self() foi uma boa ideia pra alguma manipulação específica.</p><p>No mais, acho importante saber como esse processo de resolução acontece.</p><p>Sempre escrevo outros artigos por aqui (Sobre NestJS e Angular) e acredito que possam despertar seu interesse. Caso queira dar uma olhada:</p><ul><li><a href="https://medium.com/@victorfjansen/providertoken-o-que-voc%C3%AA-mais-usa-e-n%C3%A3o-sabe-e97830ada19d">ProviderToken: O que você mais usa e não sabe</a></li><li><a href="https://medium.com/@victorfjansen/carregando-servi%C3%A7os-com-lazy-load-no-angular-668864d9f8e0">Carregando Serviços com Lazy Load no Angular</a></li><li><a href="https://medium.com/@victorfjansen/class-validators-adicionando-seguran%C3%A7a-%C3%A0-sua-aplica%C3%A7%C3%A3o-nestjs-a4d708f884b5">Class-validators — Adicionando segurança à sua aplicação NestJS</a></li><li><a href="https://medium.com/@victorfjansen/injection-scopes-carregamento-din%C3%A2mico-de-servi%C3%A7os-no-nestjs-99f8ad691e37">Injection Scopes — carregamento dinâmico de serviços no NestJS?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b30f75735f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>