<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Michalkowal on Medium]]></title>
        <description><![CDATA[Stories by Michalkowal on Medium]]></description>
        <link>https://medium.com/@michalkowal567?source=rss-59485644f376------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Iie6mDyXpia92XQpP8PE4Q.png</url>
            <title>Stories by Michalkowal on Medium</title>
            <link>https://medium.com/@michalkowal567?source=rss-59485644f376------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 18:06:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@michalkowal567/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Open source security GO project — Waffle]]></title>
            <link>https://medium.com/@michalkowal567/open-source-security-go-project-waffle-d51ebf587759?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/d51ebf587759</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 10:54:20 GMT</pubDate>
            <atom:updated>2024-03-25T10:54:20.768Z</atom:updated>
            <content:encoded><![CDATA[<h3>Open source security GO project — Waffle</h3><p><strong>Waffle looks for contributors.</strong> It could be you, as long as you are interested into software development, writing documentation or basically creating stuff that others loves to use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*56iCY2rT6aULRj3f.png" /><figcaption>The Waffle logo</figcaption></figure><h4>🤨 Web Application Firewall (WAF)— whats that?</h4><p>This is the firewall, but for the web applications. It can be on the application side but also it can be provided as the reverse proxy, which validates each request.</p><h4>🔥 WAFFLE — the WAF</h4><p><a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a> — if you don’t want to contribute, come and star, it motivates me and others to develop this ⭐️.</p><p>I have created the WAFFLE to learn how to write a proper WAF in GO. The amount of code and features is huge, that’s why I’m looking for the help from people that would love to learn GO, security or basically want to became open source contributors.</p><p>So why I started this project?</p><blockquote>A little bit than over a month ago I was asked by the recruiter: do you know how to write WAF? I didn’t know at the time. I have googled it and found some information about whats that, what it’s used for. I have built some context, but there weren’t any tutorials how to write one. I started reading a lot of whitepapers and found out, how it protects from DDOS, XSS, SQLI and other attacks.</blockquote><p>Currently the project is POC, which I’m constantly developing each day (I’m also taking part in contest: 100 commits).</p><h4>📝 What I have learned and you can expect to learn?</h4><p>This is the list:</p><ul><li>Networking</li><li>DDOS protection</li><li>XSS protection</li><li>SQL Injection protection</li><li>Machine Learning</li><li>Expression Trees</li><li>Tokenizer</li><li>YAML</li><li>How you can quit vim</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*rkG0iSrLP3dYZLqz.jpg" /></figure><p>For real, I have started learning VIM as this is the best IDE software developer can have, not only from the productiveness perspective but from the amount of plugins people are creating and the possibility to use Lua to create your IDE.</p><p>I really love to share knowledge and help others to get it in order to build <strong>the community.</strong></p><h4>The community</h4><p>I want to create the community in which we are going to develop this project as well as other great ideas you will have.</p><p>Discord: <a href="https://discord.gg/e7XU92Ba?fbclid=IwAR17xaE_D0nrnF5e3qo19cLYJMTZ2krAoL_FSzLL4-5H91vL10IU_EakKtg">https://discord.gg/e7XU92Ba</a></p><p>Join and say hello, introduce yourself and describe your motivations, it’s a nice feeling to meet new people that are motivated to make world a better place.</p><h4>The repository</h4><p><a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a> ⭐️ (come and star, and if you want to contribute, firstly join discord)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d51ebf587759" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[⏱ 20 days later. Web App Firewall in GO.]]></title>
            <link>https://medium.com/@michalkowal567/20-days-later-web-app-firewall-in-go-37913f101e02?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/37913f101e02</guid>
            <category><![CDATA[education]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Tue, 19 Mar 2024 18:22:32 GMT</pubDate>
            <atom:updated>2024-03-20T06:56:46.297Z</atom:updated>
            <content:encoded><![CDATA[<p>20 days ago I started writing an open-source implementation of the WAF. I have learned too much during that period and I want to share the knowledge with you.</p><p>btw. the project is open source and it&#39;s called a waffle.</p><p>Meet <a href="https://github.com/cebilon123/waffle"><strong>Waffle</strong></a><strong> </strong>⬅ ⭐ (star it please, it motivates me a lot 🔥).</p><p>This is <strong>WAFFLE </strong>as well:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*HCCdftvUgCz2dEF7.png" /><figcaption>Cute little gopher eating waffles, so real, for real</figcaption></figure><h4>Beginning of the story</h4><p>Some time ago I was asked: if I knew how to write the WAF. You probably guessed it, I didn’t 😥. Even more: I didn’t know what the WAF was. A quick google and I find out: web application firewall.</p><p>That’s all, bye. JK!</p><p>I like to learn new things and trust me, this thing was completely foreign for me, mainly because it’s strongly related to security and the only security I was implementing before was some simple k8s load balancing, rate limiting, and other stuff that software developers tend to do.</p><p>I googled even more, and oh boy, I didn’t know what to Google, so I slowly started to find out what it was all about. The very first example of a company that provides WAF is a <a href="https://www.cloudflare.com/application-services/products/waf/">Cloudflare</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/0*j2q9LMnxFQs3cOhK" /></figure><p>I found a lot of the articles and whitepapers regarding this subject (all of those are inside readme.md in the GitHub repository I have linked before).</p><p>In short:</p><blockquote>Web Application Firewall is like a real firewall, except it’s for web applications. The most common implementation of it is a reverse proxy with defense functionalities. It protects from DDOS attacks, XSS, and SQL Injections.</blockquote><h4>📚 What I have learned?</h4><p>btw. it&#39;s a nice thing to write down things you learn everyday, it motivates you and shows how much power you got in terms of knowledge 💪.</p><p>Those are keywords with short descriptions, you can use those if you want to learn the same things I did.</p><p>✔ <strong>WAF </strong>— As I described before, I have learned what is and how it works.</p><p>✔ <strong>DDOS</strong></p><p>In the past, I already knew how it works, but now I have exactly learned. To protect from DDOS you can use a few tools:</p><ul><li><strong>mathematical algorithms</strong>: those are the simplest ones, it’s based on the probability of packet size for both clients and attackers.</li><li><strong>ML algorithms: </strong>this is a better approach as it has a higher chance of blocking DDOS. As the administrator you still need to show to the algorithm real DDOS attacks, but it makes things a lot easier than writing custom mathematical equations.</li><li><strong>TLS fingerprinting: </strong>this is my favorite approach. It’s based on the <strong>JE3 algorithm </strong>found out by the Salesforce team around 2016. Each request, done using HTTPS protocol sends information on how the data should be encrypted by which algorithms, there are a lot of those:</li></ul><pre>var (<br> ciphers = []uint16{<br>  // TLS 1.3<br>  tls.TLS_AES_128_GCM_SHA256,<br>  tls.TLS_AES_256_GCM_SHA384,<br>  tls.TLS_CHACHA20_POLY1305_SHA256,<br><br>  // ECDSA is about 3 times faster than RSA on the server side.<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // RSA is slower on the server side but still widely used.<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // Added so all ciphers are available<br><br>  tls.TLS_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_FALLBACK_SCSV,<br> }<br><br> http11    = &quot;http/1.1&quot;<br> http2     = &quot;h2&quot;<br> http3     = &quot;h3&quot;<br> alpnProto = &quot;acme-tls/1&quot;<br>)</pre><p>The validation happens on the TLS Handshake. A lot more you can read about it by simply Googling it.</p><p>✔ <strong>XSS / SQL Injection</strong></p><p>Those are catchy ones. In the reverse proxy you can verify only the <strong>HTML XSS </strong>(the js XSS needs to be validated on the frontend side)<strong>, </strong>which means that you are validating the incoming request body by the set of rules, if the given body contains HTML, you are simply blocking it.</p><p>Did you know?💡</p><p>You can obscure a JS in order to make life harder for the protection mechanisms. I.e. this js:</p><pre>function hi() {<br>  console.log(&quot;Hello World!&quot;);<br>}<br>hi();</pre><p>can be obfscured into:</p><pre>(function(_0x195237,_0x42b798){var _0x3988f1=_0x4a33,_0x10c68b=_0x195237();while(!![]){try{var _0x3b1168=parseInt(_0x3988f1(0xc9))/0x1*(parseInt(_0x3988f1(0xce))/0x2)+parseInt(_0x3988f1(0xc7))/0x3+parseInt(_0x3988f1(0xcc))/0x4*(parseInt(_0x3988f1(0xc5))/0x5)+-parseInt(_0x3988f1(0xcf))/0x6*(parseInt(_0x3988f1(0xcd))/0x7)+parseInt(_0x3988f1(0xc8))/0x8+-parseInt(_0x3988f1(0xca))/0x9+-parseInt(_0x3988f1(0xd0))/0xa;if(_0x3b1168===_0x42b798)break;else _0x10c68b[&#39;push&#39;](_0x10c68b[&#39;shift&#39;]());}catch(_0x2019dd){_0x10c68b[&#39;push&#39;](_0x10c68b[&#39;shift&#39;]());}}}(_0x24c5,0x40214));function hi(){var _0x3abeeb=_0x4a33;console[_0x3abeeb(0xc6)](_0x3abeeb(0xcb));}function _0x4a33(_0x1be9ff,_0x31cc21){var _0x24c57a=_0x24c5();return _0x4a33=function(_0x4a33d1,_0x1167fd){_0x4a33d1=_0x4a33d1-0xc5;var _0xacd342=_0x24c57a[_0x4a33d1];return _0xacd342;},_0x4a33(_0x1be9ff,_0x31cc21);}function _0x24c5(){var _0x4c2435=[&#39;8661FMvYDC&#39;,&#39;542286oyIhHF&#39;,&#39;Hello\x20World!&#39;,&#39;57076QDwCZR&#39;,&#39;169225RvklhY&#39;,&#39;80Htuddo&#39;,&#39;78HWmmTR&#39;,&#39;7122990itgADv&#39;,&#39;140jQriRh&#39;,&#39;log&#39;,&#39;295680ZvUEBW&#39;,&#39;4039776hqprzh&#39;];_0x24c5=function(){return _0x4c2435;};return _0x24c5();}hi();</pre><p>SLQ Injection is a really rare thing, and protecting against it is a really hard thing because SQL contains so many keywords, functions, and ways how to write it.</p><p>✔ <strong>Tokenizer &amp; Expression trees</strong></p><p>As this is an open-source project and I wanted to learn new stuff I also created a custom request validation rules builder based on the configuration:</p><pre>rules:<br>  custom:<br>    - name: &quot;request payload must contain at least one character and should contain at least one header&quot;<br>      predicate: &quot;p =&gt; LEN(p.payload) &gt; 0 &amp;&amp; LEN(p.headers) &gt; 0&quot;<br>    - name: &quot;payload must be a json&quot;<br>      predicate: &quot;p =&gt; FORMAT(p.payload) == &#39;json&#39;&quot;</pre><p>The important part is the predicate. It’s strongly based on the C# semantics if it comes to the lambda expressions.</p><p>I have learned how to build tokenizer (like in the programming language compilers) and then how to build expression trees based on this compiler (it’s worth checking: <a href="https://github.com/cebilon123/waffle"><strong>Waffle</strong></a><strong> </strong>⬅ ⭐)</p><p>✔ <strong>CA certificates &amp; server certificates</strong></p><p>I have learned what is CA certificate, is and how it works. I have also learned what the Certificate Authority is, in order to sign the server certificate.</p><p>The most known CAs are:</p><ul><li>Let’s Encrypt</li><li>Godaddy</li></ul><p>and other things. There is a <a href="https://www.hs-schmalkalden.de/en/university/faculties/faculty-of-electrical-engineering/studium/use-of-it/install-ca-certificates-on-linux-systems"><strong>Linux </strong></a>package that contains the whole collection of those CAs and trust me: there is a lot.</p><h4><strong>🧡 Conclusion</strong></h4><p>To be honest, being this productive open-source developer made me learn so many new things (that I didn’t learn past the last year), which made me so much more motivated to continue what I’m doing and to learn new things.</p><p>Come and learn with me (or at least ⭐ this sweet gopher with those little waffles 🤣)</p><p><a href="https://github.com/cebilon123/waffle"><strong>Waffle</strong></a><strong> </strong>⬅ ⭐</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37913f101e02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building tokenizer  and expression tree  in golang #1]]></title>
            <link>https://medium.com/@michalkowal567/building-tokenizer-and-expression-tree-in-golang-1-eeaed03037ae?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/eeaed03037ae</guid>
            <category><![CDATA[web-application-firewall]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[compilers]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Mon, 18 Mar 2024 09:51:36 GMT</pubDate>
            <atom:updated>2024-03-18T09:51:36.618Z</atom:updated>
            <content:encoded><![CDATA[<p>At least you had written at least one line of code in your life, but have you thought how this code became the working program?</p><p>I’m creating the t<strong>okenizer and expression tree</strong> to build a predicates that will be used to validate requests in my new open source <strong>Web Application Firewall</strong>: <a href="https://github.com/cebilon123/waffle">Waffle</a>.</p><p>This is short 2 parts series that will give you some idea how things are working.</p><p>Hopefully you are like a Morty and...</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fgk3R16JhLP8RUka2nD%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExeDY1M2p3aGRvYWgzemVha3lpcHB6dWJuNWFwaDB4azVtc3Fhc2N4ZyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2Fgk3R16JhLP8RUka2nD%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZGtmMnBxY2kwaW1ucXIwM2g1ZGRucnpxdDVldW1sajl2Z3Q0MXV2cCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2Fgk3R16JhLP8RUka2nD%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="302" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8986538e5dd498f374f1c308ede37a1/href">https://medium.com/media/d8986538e5dd498f374f1c308ede37a1/href</a></iframe><h4>Expression Tree</h4><blockquote>A <strong>binary expression tree</strong> is a specific kind of a <a href="https://en.wikipedia.org/wiki/Binary_tree">binary tree</a> used to represent <a href="https://en.wikipedia.org/wiki/Expression_(mathematics)">expressions</a>. Two common types of expressions that a binary expression tree can represent are <a href="https://en.wikipedia.org/wiki/Algebra">algebraic</a> and <a href="https://en.wikipedia.org/wiki/Boolean_algebra">boolean</a>. These trees can represent expressions that contain both <a href="https://en.wikipedia.org/wiki/Unary_operation">unary</a> and <a href="https://en.wikipedia.org/wiki/Binary_function">binary</a> operators.</blockquote><blockquote>Like any binary tree, each node of a binary expression tree has zero, one, or two children. This restricted structure simplifies the processing of expression trees.</blockquote><p>In the simpler way:</p><p>The expression tree is a representation of the predicate that will be executed over a source of data. This is used i.e. in MongoDB golang driver query language to create the possibility to fetch data based on the dynamic predicates.</p><h4>The problem</h4><p>As previously mentioned this blog post is based on my WAF project which is available on github, come and ⭐️ it :)</p><p><a href="https://github.com/cebilon123/waffle">Waffle</a></p><p>In the WAF I had a nice idea for the feature: I want to create some custom rules from configuration to validate requests. This is example of those rules:</p><pre>rules:<br>  custom:<br>    - name: &quot;request payload must contains at least one character and should contain at least one header&quot;<br>      predicate: &quot;p =&gt; LEN(p.payload) &gt; 0 &amp;&amp; LEN(p.headers) &gt; 0&quot;<br>    - name: &quot;payload must be a json&quot;<br>      predicate: &quot;p =&gt; FORMAT(p.payload) == &#39;json&#39;&quot;</pre><p>The idea is simple: we are having this predicate (strongly based on the C# ones) to validate requests against it, it looks clean and simple, but how to create the code that will be validating the requests based on those predicates?</p><p>Here comes the expression trees with the help.</p><pre>package rule<br><br>import &quot;waffle/internal/request&quot;<br><br>type node interface {<br> Eval(r request.Wrapper) bool<br>}<br><br>type and struct {<br> left, right node<br>}<br><br>func (a and) Eval(r request.Wrapper) bool {<br> return a.left.Eval(r) &amp;&amp; a.right.Eval(r)<br>}<br><br>type or struct {<br> left, right node<br>}<br><br>func (o or) Eval(r request.Wrapper) bool {<br> return o.left.Eval(r) || o.right.Eval(r)<br>}<br><br>type eq struct {<br> left, right node<br>}<br><br>func (e eq) Eval(r request.Wrapper) bool {<br> return e.left.Eval(r) == e.right.Eval(r)<br>}<br><br>type gt struct {<br> valueFunc func(r request.Wrapper) (int, error)<br> check     int<br>}<br><br>func (g gt) Eval(r request.Wrapper) bool {<br> value, err := g.valueFunc(r)<br> if err != nil {<br>  return false<br> }<br><br> return value &gt; g.check<br>}</pre><p>I have a node interface that forces structs to implement method eval, that takes the <em>request.Wrapper (this struct wraps native golang request) </em>as the parameter and returns bool. This is implemented by all the expression nodes:</p><ul><li>and</li><li>or</li><li>eq — equal</li><li>gt — greater</li></ul><p>Whats even better: it can be used to create new rules as the system will grow. So, based on the expression nodes we can translate this predicate:</p><pre>predicate: &quot;p =&gt; LEN(p.payload) &gt; 0 &amp;&amp; LEN(p.headers) &gt; 0&quot;</pre><p>into:</p><pre> tree := and{<br> left: gt{<br>  valueFunc: func(r request.Wrapper) (int, error) {<br>   bytes, err := io.ReadAll(r.Request.Body)<br>   if err != nil {<br>    return 0, fmt.Errorf(&quot;cannot read bytes: %w&quot;, err)<br>   }<br> <br>   return len(bytes), nil<br>  },<br>  check: 0,<br> },<br> right: gt{<br>  valueFunc: func(r request.Wrapper) (int, error) {<br>   return len(r.Request.Header), nil<br>  },<br>  check: 0,<br> },<br> }</pre><p>As you can see, on the top level of the tree is an <strong>and</strong>, which divides two parts of the expression: the payload length check and the headers length check. Then we are simply using the <strong>greater (gt) </strong>node to create the desired predicates.</p><p>Whats even better: the tree is a node itself so it can be even bigger, as long as you have some RAM to spare.</p><p>We can then basically evaluate this tree with the usage of the request.Wrapper. So if we are going to have i.e. 100 rules in the configuration we can create 100 expression trees based on that and use those to evaluate some logical expression on the requests.</p><h4>Conclusion</h4><p>The expression trees are overall simple, but the hard part is how to translate this predicate:</p><pre>predicate: &quot;p =&gt; LEN(p.payload) &gt; 0 &amp;&amp; LEN(p.headers) &gt; 0&quot;</pre><p>Into this expression tree:</p><pre>tree := and{<br> left: gt{<br>  valueFunc: func(r request.Wrapper) (int, error) {<br>   bytes, err := io.ReadAll(r.Request.Body)<br>   if err != nil {<br>    return 0, fmt.Errorf(&quot;cannot read bytes: %w&quot;, err)<br>   }<br> <br>   return len(bytes), nil<br>  },<br>  check: 0,<br> },<br> right: gt{<br>  valueFunc: func(r request.Wrapper) (int, error) {<br>   return len(r.Request.Header), nil<br>  },<br>  check: 0,<br> },<br> }</pre><p>To do such a thing we are going to create tokenizer, that is based on the idea of programming language tokenizers, that are the heart of the programming language compilers.</p><p>Follow me, star a github project and stay for the part 2, which will be written after I create the tokenizer in the github repository 😃</p><p>See you next time!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*5ZkuRcqlkVkZtZ6B.jpg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eeaed03037ae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reusable in-memory generic cache in Golang ]]></title>
            <link>https://blog.stackademic.com/reusable-in-memory-generic-cache-in-golang-89979c1f75a6?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/89979c1f75a6</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[clean-code]]></category>
            <category><![CDATA[code]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Sun, 10 Mar 2024 08:53:43 GMT</pubDate>
            <atom:updated>2024-03-12T01:58:16.842Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*YUdaMx_8D_Q5UowU.jpg" /><figcaption><a href="https://softwarelab.org/blog/what-is-a-cache/">https://softwarelab.org/blog/what-is-a-cache/</a></figcaption></figure><p>I will show you, how I created a generic, easy-to-re-use in-memory cache in Golang, which is fast and quick to develop.</p><p>I will also give you some tips regarding the functionality and how to modify it, so take notes!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FxT9IgtvIZB1w7dVO6s%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExaTZta2JoMXh1bHE0c2R2MWE5MXR1bG5zYzY1dmxzNjJ2bzFnbDltOCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FxT9IgtvIZB1w7dVO6s%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExNHp6Y2I1NjZjNWViZzN3M3JqN3VscW53MnEyaGQ4YTBnYzA1eW9hYyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FxT9IgtvIZB1w7dVO6s%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="435" frameborder="0" scrolling="no"><a href="https://medium.com/media/11b5c9da65a3be8aa2a27e711f5a35fc/href">https://medium.com/media/11b5c9da65a3be8aa2a27e711f5a35fc/href</a></iframe><h4>❔ What are we going to build exactly?</h4><p>We will build an in-memory cache in golang (which means that we will store entries in RAM).</p><p>So we will build an in-memory Redis clone if you need more context.</p><h4>💫 Prerequisites</h4><p>This is the list of things that you will need to create the cache:</p><ul><li>GO</li><li><a href="https://github.com/patrickmn/go-cache">go-cache package</a></li></ul><h4>🔥 The solution</h4><p>The solution comes from my open-source project &quot;Waffle&quot; which is web application firewall written in go. If you are curious about the implementation, and some security subjects: star it and follow ✨</p><p>📘<a href="https://github.com/cebilon123/waffle">Waffle</a>📘</p><p>Now let’s start with the implementation.</p><p>First of all, create a package cache, it will be used to contain the domain of the cache that we are building.</p><pre>package cache<br></pre><p>Now we can import the packages we are going to use:</p><ul><li>the std time</li><li>the go-cache mentioned above</li></ul><pre>package cache<br><br>import (<br> &quot;time&quot;<br><br> &quot;github.com/patrickmn/go-cache&quot;<br>)</pre><blockquote>The <em>std time</em> package will be needed to pass the lifespan duration of cache entries</blockquote><p>Let’s start with the struct outline, we are going to call it cache, and we are going to use generics.</p><pre>package cache<br><br>import (<br> &quot;time&quot;<br><br> &quot;github.com/patrickmn/go-cache&quot;<br>)<br><br>type Cache[K any, T any] struct {<br> cache *cache.Cache<br>}</pre><p>Two generic types are passed to this struct, <strong>the K</strong> and <strong>the T</strong>. K is the shortcut for the K=key, T=Value.</p><p>Now let’s add the constructor:</p><pre>package cache<br><br>import (<br> &quot;time&quot;<br><br> &quot;github.com/patrickmn/go-cache&quot;<br>)<br><br>type Cache[K any, T any] struct {<br> cache *cache.Cache<br>}<br><br>func New[K any, T any](defaultExpiration time.Duration, cleanupInterval time.Duration) *Cache[K, T] {<br> c := cache.New(defaultExpiration, cleanupInterval)<br><br> return &amp;Cache[K, T]{<br>  cache: c,<br> }<br>}</pre><p>As you can see, we are simply creating the cache from the go-cache package and we are passing some expirations.</p><p>Now we can add methods for setting the value and getting the value</p><pre>package cache<br><br>import (<br> &quot;time&quot;<br><br> &quot;github.com/patrickmn/go-cache&quot;<br>)<br><br>type Cache[K any, T any] struct {<br> cache *cache.Cache<br>}<br><br>func New[K any, T any](defaultExpiration time.Duration, cleanupInterval time.Duration) *Cache[K, T] {<br> c := cache.New(defaultExpiration, cleanupInterval)<br><br> return &amp;Cache[K, T]{<br>  cache: c,<br> }<br>}<br><br>func (c *Cache[K, T]) Set(key K, val any) {<br> c.cache.Set(string(key), val, cache.DefaultExpiration)<br>}<br><br>func (c *Cache[K, T]) Get(key K) (*T, bool) {<br> v, ok := c.cache.Get(string(key))<br> if !ok {<br>  return nil, false<br> }<br><br> vTyped, ok := v.(T)<br> if !ok {<br>  return nil, false<br> }<br><br> return &amp;vTyped, true<br>}</pre><p>The Set method is simple: we are passing the key and the value we want to cache, but wait! There is a problem: go-cache uses a string as the key for the cache entry, and our generic type is any, so the compiler wouldn’t let us use it here. We need to create a custom constraint</p><blockquote>The custom constraint is an interface where we describe what types of structs we can use as the generic parameter.</blockquote><p>We are going to create this constraint:</p><pre>type stringConstraint interface {<br> ~string<br>}</pre><p><strong>The “~”:</strong></p><blockquote>For type constraints we usually don’t care about a specific type, such as string; we are interested in all string types. That is what the ~ token is for. The expression ~string means the set of all types whose underlying type is string. This includes the type string itself as well as all types declared with definitions such as type MyString string</blockquote><p>Rephrasing the text above: we can basically create a custom type that is a string and use it here as well as shown below:</p><pre>type customKey string</pre><p>So now our code looks like this:</p><pre>package cache<br><br>import (<br> &quot;time&quot;<br><br> &quot;github.com/patrickmn/go-cache&quot;<br>)<br><br>type stringConstraint interface {<br> ~string<br>}<br><br>type Cache[K stringConstraint, T any] struct {<br> cache *cache.Cache<br>}<br><br>func New[K stringConstraint, T any](defaultExpiration time.Duration, cleanupInterval time.Duration) *Cache[K, T] {<br> c := cache.New(defaultExpiration, cleanupInterval)<br><br> return &amp;Cache[K, T]{<br>  cache: c,<br> }<br>}<br><br>func (c *Cache[K, T]) Set(key K, val any) {<br> c.cache.Set(string(key), val, cache.DefaultExpiration)<br>}<br><br>func (c *Cache[K, T]) Get(key K) (*T, bool) {<br> v, ok := c.cache.Get(string(key))<br> if !ok {<br>  return nil, false<br> }<br><br> vTyped, ok := v.(T)<br> if !ok {<br>  return nil, false<br> }<br><br> return &amp;vTyped, true<br>}</pre><p><strong>And that&#39;s the final form of the cache</strong>. So you can copy it if you want, or keep reading to find out more.</p><p>The Get method looks for a given entry in the cache by the usage of our key, if it finds one, we are then doing the <strong>type assertion </strong>to our desired type T, and finally, we are returning it from the method.<br>Of course, if the entry isn’t found or the entry isn’t of type T, we are returning false.</p><h4>✅ Conclusion</h4><p>The creation of a custom generic cache isn’t that hard, to be honest, it’s pretty simple and usage of such a thing is much cleaner than using the raw package.</p><p>If you 💛 what I create, follow me here, on Medium and of course check out my open-source project that the code example is from:</p><p><a href="https://github.com/cebilon123/waffle">GitHub - cebilon123/waffle: Web Application Firewall, made in go.</a></p><p>See you next time!</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=89979c1f75a6" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/reusable-in-memory-generic-cache-in-golang-89979c1f75a6">Reusable in-memory generic cache in Golang 📚</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[‍♀️ ‍♀️ ‍♀️ Multi clients reverse proxy in golang.]]></title>
            <link>https://blog.devgenius.io/%EF%B8%8F-%EF%B8%8F-%EF%B8%8F-multi-clients-reverse-proxy-in-golang-bc9614eae113?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc9614eae113</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Wed, 06 Mar 2024 08:40:24 GMT</pubDate>
            <atom:updated>2024-03-11T17:13:47.864Z</atom:updated>
            <content:encoded><![CDATA[<h3>🚴‍♀️ 🚴‍♀️ 🚴‍♀️ Multi hosts reverse proxy in golang.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1021/1*ntCaWmLjCjMRHcQn1ZwI7Q.png" /></figure><p>Hey there 👋</p><p>You are here because you were interested in building <strong>the reverse proxy in golang</strong>, that’s a good place to find out how to make one.</p><p>However, it’s not simple <strong>single client reverse proxy</strong>, but <strong>multi clients reverse proxy </strong>and even more, I will show you how to write clean one.</p><p>The creation of such a thing in golang isn’t hard.</p><p>You just need coffe and some time, and I will take you on this knowledge gaining adventure!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2F1hXY6iNdTFpTW4je85%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExazJxMDdtdHlqanNtdzVwYWFnNzBlZDZucTdnbWk5bXp0Z2d2Z2Z1cyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2F1hXY6iNdTFpTW4je85%2Fgiphy-downsized-large.gif&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExbm8yM2YwMTNsZHhwZmJ1eDBpeGtiZDVrcXEyeHlvNXExanh3cHJ6NSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2F1hXY6iNdTFpTW4je85%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="435" frameborder="0" scrolling="no"><a href="https://medium.com/media/d2cf7129eb6dfd2ef668911e7e89b2c8/href">https://medium.com/media/d2cf7129eb6dfd2ef668911e7e89b2c8/href</a></iframe><p><strong>Dislaimer:</strong> The educational value you will get from this post is huge, but what’s important: all examples are based on the open source project worth to see and ⭐️</p><blockquote><a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a></blockquote><p>The project is pretty new, and if you want to try your software development powers, you should talk with author (me 😂).</p><h3>Architecture of the solution</h3><p>As mentioned previously, the code is based on the open source project, that I will try to explain to you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/1*XKPKmEGJEObfxBC3pYMXLQ.png" /><figcaption>Package dependencies graph</figcaption></figure><p>The solution has been a little bit abstracted, but that are a good practices to scale project to the moon 🚀 💪</p><p>Our root, the part that the whole post is focused on is: <strong>Proxy</strong> package. Inside the proxy packages we are using the <strong>certificate and redirect packages</strong>, and then the redirect implements others, less important for the whole solution but still worth mentioning.</p><p>Now I will explain to you some packages and the code written there.</p><h4>🔐 The config package</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/240/1*CL9aBjw-iwUlcRXfT79JRw.png" /><figcaption>File structure of the package</figcaption></figure><p>The important file in this package is the <em>yaml.go.</em></p><pre>package config<br><br>import (<br> &quot;fmt&quot;<br><br> &quot;github.com/goccy/go-yaml&quot;<br>)<br><br>type YamlConfig struct {<br> DNS []*Dns `yaml:&quot;dns&quot;`<br>}<br><br>type Dns struct {<br> Host    string `yaml:&quot;host&quot;`<br> Address string `yaml:&quot;address&quot;`<br>}<br><br>func NewYamlConfig(yamlBytes []byte) (*YamlConfig, error) {<br> var yamlCfg YamlConfig<br> if err := yaml.Unmarshal(yamlBytes, &amp;yamlCfg); err != nil {<br>  return nil, fmt.Errorf(&quot;unmarshal yaml: %w&quot;, err)<br> }<br><br> return &amp;yamlCfg, nil<br>}</pre><p>As you can see, the file is simple. This is the structure, that is being used in the process of unmarshaling the yaml config, which then can be used to have easy access to the configuration of the server.</p><p>This is example of the yaml mapped to this struct:</p><pre>dns:<br>  - host: &quot;google.localhost:8080&quot;<br>    address: &quot;https://google.com&quot;<br>  - host: &quot;100commitow.localhost&quot;<br>    address: &quot;https://100commitow.pl&quot;</pre><p>This is a collection of dns, and the second host wont work, because there isn&#39;t a port provided. If you would have real domain, you can use it here, earlier but you would need to update i.e. A and AAA records</p><p>This yaml config is then later used i.e. in the domain package, which you can notice below.</p><h4>📚 The domain package</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/1*1EfYUh11fkEgLUX8sgYmJg.png" /><figcaption>File structure of the package</figcaption></figure><p>The domain is basically our internal dns, which then will be used to help in redirection of the incoming requests to the desired servers.</p><p><em>The dns.go:</em></p><pre>// Package domain contains all the code and logic<br>// related to working with the internal domain system<br>package domain<br><br>import &quot;net/url&quot;<br><br>// NameSystemProvider provides address of the destination for given<br>// registered domain.<br>type NameSystemProvider interface {<br> GetAddress(host string) (*url.URL, error)<br>}</pre><p>There is even a description of the package as well as the interface that is created inside the package. Given interface is then later used inside <em>redirect handler</em> inside <strong>redirect package.</strong></p><p>Creation of this interface creates the abstraction over implementation of the provider. What I mean by that: imagine you are starting developing the project, and to simplify things you are configuring the dns from the yaml config file, with that being said you can have <em>YamlNameSystemProvider, </em>that implements this interface. This is what author did:</p><p><em>YamlNameSystemProvider.go</em></p><pre>package domain<br><br>import (<br> &quot;fmt&quot;<br> &quot;net/url&quot;<br><br> &quot;waffle/internal/config&quot;<br>)<br><br>type YamlNameSystemProvider struct {<br> cfg *config.YamlConfig<br>}<br><br>func NewYamlNameSystemProvider(cfg *config.YamlConfig) *YamlNameSystemProvider {<br> return &amp;YamlNameSystemProvider{<br>  cfg: cfg,<br> }<br>}<br><br>func (y *YamlNameSystemProvider) GetAddress(host string) (*url.URL, error) {<br> for _, h := range y.cfg.DNS {<br>  if h.Host == host {<br>   urlAddress, err := url.Parse(h.Address)<br>   if err != nil {<br>    return nil, fmt.Errorf(&quot;parse dns address: %w&quot;, err)<br>   }<br><br>   return urlAddress, nil<br>  }<br> }<br><br> return nil, fmt.Errorf(&quot;&#39;%s&#39; not found in configuration&quot;, host)<br>}</pre><p>As you can see, the implementation isn’t that hard, and usage of the interfaces helped in achieving this legendary level of abstraction and usability of the solution.</p><p>Given struct implements the interface by having this method with the receiver:</p><pre>func (y *YamlNameSystemProvider) GetAddress(host string) (*url.URL, error) {<br> for _, h := range y.cfg.DNS {<br>  if h.Host == host {<br>   urlAddress, err := url.Parse(h.Address)<br>   if err != nil {<br>    return nil, fmt.Errorf(&quot;parse dns address: %w&quot;, err)<br>   }<br><br>   return urlAddress, nil<br>  }<br> }<br><br> return nil, fmt.Errorf(&quot;&#39;%s&#39; not found in configuration&quot;, host)<br>}</pre><p>It basically goes through collection of dns, and if dns with given host is found, it <strong>returns it’s address as the url.Url</strong>.</p><h4><strong>🔄 The redirect package</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/210/1*Eb5fx4F8AgwufZt6Hd7rag.png" /><figcaption>File structure of the package</figcaption></figure><p>As you probably guessed by the name, the redirect package is responsible for redirecting incoming requests.</p><pre>package redirect<br><br>import (<br> &quot;fmt&quot;<br> &quot;net/http&quot;<br> &quot;net/http/httputil&quot;<br> &quot;strings&quot;<br> &quot;time&quot;<br> <br> &quot;waffle/internal/domain&quot;<br>)<br><br>type Handler struct {<br> dns domain.NameSystemProvider<br>}<br><br>func NewHandler(dns domain.NameSystemProvider) *Handler {<br> return &amp;Handler{<br>  dns: dns,<br> }<br>}<br><br>var _ http.Handler = (*Handler)(nil)<br><br>func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {<br> addr, err := h.dns.GetAddress(r.Host)<br> if err != nil {<br>  w.WriteHeader(http.StatusBadGateway)<br>  return<br> }<br><br> proxy := httputil.NewSingleHostReverseProxy(addr)<br><br> r.URL.Host = addr.Host<br> r.URL.Scheme = addr.Scheme<br> r.Header.Set(&quot;X-Forwarded-Host&quot;, r.Header.Get(&quot;Host&quot;))<br> r.Host = addr.Host<br> //trim reverseProxyRoutePrefix<br> path := r.URL.Path<br> r.URL.Path = strings.TrimLeft(path, addr.Path)<br> fmt.Printf(&quot;[ TinyRP ] Redirecting request to %s at %s\n&quot;, r.URL, time.Now().UTC())<br> proxy.ServeHTTP(w, r)<br>}</pre><p>The Handler struct:</p><p>Handler struct implements the http.Handler interface, which can be confirmed by this line</p><pre>var _ http.Handler = (*Handler)(nil)</pre><p>This implementation basically tells that Handler can be assigned to the http.Handler and with that being said it also means that the handler implements http.Handler. If that wouldn&#39;t happen, go would return an error: that given Handler isn&#39;t implementing the http.Handler</p><p>Also, the Handler struct uses the abstraction that was mentioned before: <em>dns.NameSystemProvider.</em></p><p>This is the implemented http.Handler ServeHTTP method:</p><pre>func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {<br> addr, err := h.dns.GetAddress(r.Host)<br> if err != nil {<br>  w.WriteHeader(http.StatusBadGateway)<br>  return<br> }<br><br> proxy := httputil.NewSingleHostReverseProxy(addr)<br><br> r.URL.Host = addr.Host<br> r.URL.Scheme = addr.Scheme<br> r.Header.Set(&quot;X-Forwarded-Host&quot;, r.Header.Get(&quot;Host&quot;))<br> r.Host = addr.Host<br> path := r.URL.Path<br> r.URL.Path = strings.TrimLeft(path, addr.Path)<br> fmt.Printf(&quot;[ TinyRP ] Redirecting request to %s at %s\n&quot;, r.URL, time.Now().UTC())<br> proxy.ServeHTTP(w, r)<br>}</pre><p>As you can see, whenever client calls given domain, it goes through this method, and in the very first line, we are using dns provider mentioned before to get the url address of desired server. Then we are doing some overwriting of host, scheme, and we are adding headers.</p><p>There is also used httputil package</p><pre>proxy := httputil.NewSingleHostReverseProxy(addr)</pre><p>Which basically proxies given request to the desired server.</p><h4>📝 The certificate package</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/229/1*QYBPNp8FFwQxMCZMhoisrw.png" /><figcaption>The file structure of package</figcaption></figure><pre>package certificate<br><br>import (<br> &quot;crypto/tls&quot;<br> &quot;crypto/x509&quot;<br> &quot;fmt&quot;<br>)<br><br>// Provider should be implemented by the structs<br>// that are able to provide certificates.<br>type Provider interface {<br> GetTLSCertificate() (*tls.Certificate, error)<br> GetCACertificatesPool() (*x509.CertPool, error)<br>}<br><br>type LocalCertificatesProvider struct {<br> customCaCerts [][]byte<br> certPEMBlock  []byte<br> keyPEMBlock   []byte<br>}<br><br>func NewLocalCertificatesProvider(caCerts [][]byte, certPEMBlock, keyPEMBlock []byte) *LocalCertificatesProvider {<br> return &amp;LocalCertificatesProvider{<br>  customCaCerts: caCerts,<br>  certPEMBlock:  certPEMBlock,<br>  keyPEMBlock:   keyPEMBlock,<br> }<br>}<br><br>func (l *LocalCertificatesProvider) GetTLSCertificate() (*tls.Certificate, error) {<br> cert, err := tls.X509KeyPair(l.certPEMBlock, l.keyPEMBlock)<br> if err != nil {<br>  return nil, fmt.Errorf(&quot;load x509 key pair: %w&quot;, err)<br> }<br><br> return &amp;cert, nil<br>}<br><br>func (l *LocalCertificatesProvider) GetCACertificatesPool() (*x509.CertPool, error) {<br> caCertPool, err := x509.SystemCertPool()<br> if err != nil {<br>  return nil, fmt.Errorf(&quot;load system ca cert pool: %w&quot;, err)<br> }<br><br> if caCertPool == nil {<br>  caCertPool = x509.NewCertPool()<br> }<br><br> if len(l.customCaCerts) &gt; 0 {<br>  for _, caCert := range l.customCaCerts {<br>   caCertPool.AppendCertsFromPEM(caCert)<br>  }<br> }<br><br> return caCertPool, nil<br>}</pre><p>This code isn’t that easy to read 👀, I know that, but what it does it’s returning the certificates. There are two types of certs returned here:</p><ul><li>CA certificate — this one is returned in the certificate pool, because there is a lot of CAs (certificate authorities) in the world. Real ones are i.e. go daddy, or let’s encrypt</li><li>Server certificate — this is our SSL/TLS certificate, that is visible in the web browser whenever we are connecting using https protocol.</li></ul><p>The thing worth mentioning is that we are creating local CA and server cert using the local CA. The makefile for that:</p><pre>certs_windows:<br> mkdir &quot;./cmd/proxy/.cert&quot;<br> # Create CA (certificate authority)<br> openssl ecparam -out ./cmd/proxy/.cert/ca.key -name prime256v1 -genkey<br> openssl req -new -sha256 -key ./cmd/proxy/.cert/ca.key -out ./cmd/proxy/.cert/ca.csr<br> openssl x509 -req -sha256 -days 3650 -in ./cmd/proxy/.cert/ca.csr -signkey ./cmd/proxy/.cert/ca.key -out ./cmd/proxy/.cert/ca.crt<br><br> # Create server certificate<br> openssl ecparam -out ./cmd/proxy/.cert/server.key -name prime256v1 -genkey<br> openssl req -new -sha256 -key ./cmd/proxy/.cert/server.key -out ./cmd/proxy/.cert/server.csr<br> openssl x509 -req -in ./cmd/proxy/.cert/server.csr -CA ./cmd/proxy/.cert/ca.crt -CAkey ./cmd/proxy/.cert/ca.key -CAcreateserial -out ./cmd/proxy/.cert/server.crt -days 3650 -sha256<br> openssl x509 -in ./cmd/proxy/.cert/server.crt -text -noout</pre><p>It creates dir, which then is being populated: firstly by the ca related stuff, and then by the server certs (signed by the CA we created before).</p><h4>🍔 The main dish (proxy)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/190/1*oIp8wUJhfuoMO2cz5vfW6A.png" /><figcaption>The file structure of the proxy package</figcaption></figure><p>I will show you the full implementation and then I will try to explain to you the parts of this code.</p><pre>package proxy<br><br>import (<br> &quot;crypto/rand&quot;<br> &quot;crypto/tls&quot;<br> &quot;errors&quot;<br> &quot;fmt&quot;<br> &quot;log&quot;<br> &quot;net/http&quot;<br> &quot;os&quot;<br> &quot;time&quot;<br><br> &quot;waffle/internal/certificate&quot;<br> &quot;waffle/internal/redirect&quot;<br>)<br><br>var (<br> ciphers = []uint16{<br>  // TLS 1.3<br>  tls.TLS_AES_128_GCM_SHA256,<br>  tls.TLS_AES_256_GCM_SHA384,<br>  tls.TLS_CHACHA20_POLY1305_SHA256,<br><br>  // ECDSA is about 3 times faster than RSA on the server side.<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // RSA is slower on the server side but still widely used.<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // Added so all ciphers are available<br><br>  tls.TLS_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_FALLBACK_SCSV,<br> }<br><br> http11    = &quot;http/1.1&quot;<br> http2     = &quot;h2&quot;<br> http3     = &quot;h3&quot;<br> alpnProto = &quot;acme-tls/1&quot;<br>)<br><br>type Server struct {<br> addr                string<br> certificateProvider certificate.Provider<br> redirectHandler     *redirect.Handler<br>}<br><br>func NewServer(<br> addr string,<br> certificateProvider certificate.Provider,<br> redirectHandler *redirect.Handler,<br>) *Server {<br> return &amp;Server{<br>  addr:                addr,<br>  certificateProvider: certificateProvider,<br>  redirectHandler:     redirectHandler,<br> }<br>}<br><br>func (s *Server) Start() error {<br> caCertPool, err := s.certificateProvider.GetCACertificatesPool()<br> if err != nil {<br>  return fmt.Errorf(&quot;get ca certificates pool using certificate provider: %w&quot;, err)<br> }<br><br> serverCertificate, err := s.certificateProvider.GetTLSCertificate()<br> if err != nil {<br>  return fmt.Errorf(&quot;get tls certificate using certificate provider: %w&quot;, err)<br> }<br><br> tlsConfig := &amp;tls.Config{<br>  MinVersion:   tls.VersionTLS13,<br>  MaxVersion:   tls.VersionTLS13,<br>  CipherSuites: ciphers,<br>  NextProtos:   []string{http3, http2, http11, alpnProto},<br>  Certificates: []tls.Certificate{*serverCertificate},<br>  ClientAuth:   tls.VerifyClientCertIfGiven,<br>  Rand:         rand.Reader,<br>  RootCAs:      caCertPool,<br>  ClientCAs:    caCertPool,<br> }<br><br> tcpListener, err := tls.Listen(&quot;tcp&quot;, s.addr, tlsConfig)<br> if err != nil {<br>  return fmt.Errorf(&quot;tls tcp listener listen: %w&quot;, err)<br> }<br><br> server := &amp;http.Server{<br>  Handler:           s.redirectHandler,<br>  Addr:              fmt.Sprintf(&quot;%s%s&quot;, &quot;localhost&quot;, s.addr),<br>  ReadHeaderTimeout: 120 * time.Second,<br>  WriteTimeout:      120 * time.Second,<br>  IdleTimeout:       120 * time.Second,<br>  ReadTimeout:       120 * time.Second,<br>  TLSConfig:         tlsConfig,<br>  MaxHeaderBytes:    1048576,<br>  ErrorLog:          log.New(os.Stderr, &quot;proxy server: &quot;, 0),<br> }<br><br> if err := server.Serve(tcpListener); err != nil &amp;&amp; !errors.Is(err, http.ErrServerClosed) {<br>  return fmt.Errorf(&quot;server serve: %w&quot;, err)<br> }<br><br> return nil<br>}</pre><p>Let’s start with the vars:</p><pre>var (<br> ciphers = []uint16{<br>  // TLS 1.3<br>  tls.TLS_AES_128_GCM_SHA256,<br>  tls.TLS_AES_256_GCM_SHA384,<br>  tls.TLS_CHACHA20_POLY1305_SHA256,<br><br>  // ECDSA is about 3 times faster than RSA on the server side.<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // RSA is slower on the server side but still widely used.<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // Added so all ciphers are available<br><br>  tls.TLS_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_FALLBACK_SCSV,<br> }<br><br> http11    = &quot;http/1.1&quot;<br> http2     = &quot;h2&quot;<br> http3     = &quot;h3&quot;<br> alpnProto = &quot;acme-tls/1&quot;<br>)</pre><p>The ciphers are the ciphers that are used whenever clients connects through reverse proxy. During TLS handshake, we are veryfing certificates send by the client, based on those ciphers.</p><pre> http11    = &quot;http/1.1&quot;<br> http2     = &quot;h2&quot;<br> http3     = &quot;h3&quot;<br> alpnProto = &quot;acme-tls/1&quot;</pre><p>Those are the protocols that our server accepts.</p><pre>type Server struct {<br> addr                string<br> certificateProvider certificate.Provider<br> redirectHandler     *redirect.Handler<br>}</pre><p>This is our server structure. It has few fields:</p><ul><li>addr — address of our server</li><li>certificateProvider — the certificate provider mentioned before</li><li>redirectHandler — this is the redirect handler mentioned before as well</li></ul><pre>func (s *Server) Start() error {<br> caCertPool, err := s.certificateProvider.GetCACertificatesPool()<br> if err != nil {<br>  return fmt.Errorf(&quot;get ca certificates pool using certificate provider: %w&quot;, err)<br> }<br><br> serverCertificate, err := s.certificateProvider.GetTLSCertificate()<br> if err != nil {<br>  return fmt.Errorf(&quot;get tls certificate using certificate provider: %w&quot;, err)<br> }<br><br> tlsConfig := &amp;tls.Config{<br>  MinVersion:   tls.VersionTLS13,<br>  MaxVersion:   tls.VersionTLS13,<br>  CipherSuites: ciphers,<br>  NextProtos:   []string{http3, http2, http11, alpnProto},<br>  Certificates: []tls.Certificate{*serverCertificate},<br>  ClientAuth:   tls.VerifyClientCertIfGiven,<br>  Rand:         rand.Reader,<br>  RootCAs:      caCertPool,<br>  ClientCAs:    caCertPool,<br> }<br><br> tcpListener, err := tls.Listen(&quot;tcp&quot;, s.addr, tlsConfig)<br> if err != nil {<br>  return fmt.Errorf(&quot;tls tcp listener listen: %w&quot;, err)<br> }<br><br> server := &amp;http.Server{<br>  Handler:           s.redirectHandler,<br>  Addr:              fmt.Sprintf(&quot;%s%s&quot;, &quot;localhost&quot;, s.addr),<br>  ReadHeaderTimeout: 120 * time.Second,<br>  WriteTimeout:      120 * time.Second,<br>  IdleTimeout:       120 * time.Second,<br>  ReadTimeout:       120 * time.Second,<br>  TLSConfig:         tlsConfig,<br>  MaxHeaderBytes:    1048576,<br>  ErrorLog:          log.New(os.Stderr, &quot;proxy server: &quot;, 0),<br> }<br><br> if err := server.Serve(tcpListener); err != nil &amp;&amp; !errors.Is(err, http.ErrServerClosed) {<br>  return fmt.Errorf(&quot;server serve: %w&quot;, err)<br> }<br><br> return nil<br>}</pre><p>Then we have this method, our core method for this blog post. Basically, we are creating tls listener as well as a server here.</p><pre>caCertPool, err := s.certificateProvider.GetCACertificatesPool()<br> if err != nil {<br>  return fmt.Errorf(&quot;get ca certificates pool using certificate provider: %w&quot;, err)<br> }<br><br> serverCertificate, err := s.certificateProvider.GetTLSCertificate()<br> if err != nil {<br>  return fmt.Errorf(&quot;get tls certificate using certificate provider: %w&quot;, err)<br> }</pre><p>In those two lines we are fetching the certifictes that are provided by the <strong>certificateProvider</strong> abstraction.</p><pre>tlsConfig := &amp;tls.Config{<br>  MinVersion:   tls.VersionTLS13,<br>  MaxVersion:   tls.VersionTLS13,<br>  CipherSuites: ciphers,<br>  NextProtos:   []string{http3, http2, http11, alpnProto},<br>  Certificates: []tls.Certificate{*serverCertificate},<br>  ClientAuth:   tls.VerifyClientCertIfGiven,<br>  Rand:         rand.Reader,<br>  RootCAs:      caCertPool,<br>  ClientCAs:    caCertPool,<br> }</pre><p>Then those certificates are used in the creation of the TLS config. Also we are using the ciphers, protocols mentioned before.</p><pre>tcpListener, err := tls.Listen(&quot;tcp&quot;, s.addr, tlsConfig)<br> if err != nil {<br>  return fmt.Errorf(&quot;tls tcp listener listen: %w&quot;, err)<br> }</pre><p>Then we are basically creating the tcp listener that will listen on the incoming requests and validate those using the tls.Config.</p><p>worth mentioning is that this tls.Config can be strongly modified. We can change how the whole tls handshake process looks like or how certificates are being verified</p><pre>server := &amp;http.Server{<br>  Handler:           s.redirectHandler,<br>  Addr:              fmt.Sprintf(&quot;%s%s&quot;, &quot;localhost&quot;, s.addr),<br>  ReadHeaderTimeout: 120 * time.Second,<br>  WriteTimeout:      120 * time.Second,<br>  IdleTimeout:       120 * time.Second,<br>  ReadTimeout:       120 * time.Second,<br>  TLSConfig:         tlsConfig,<br>  MaxHeaderBytes:    1048576,<br>  ErrorLog:          log.New(os.Stderr, &quot;proxy server: &quot;, 0),<br> }<br><br> if err := server.Serve(tcpListener); err != nil &amp;&amp; !errors.Is(err, http.ErrServerClosed) {<br>  return fmt.Errorf(&quot;server serve: %w&quot;, err)<br> }</pre><p>Then we are creating simple http server, that has been already created in 1000 of other code examples. The difference is the usage of the <strong>redirectHandler</strong> created before, as well as “serving” using the <strong>tcpListener</strong> we did create.</p><p>Creation of the <strong>TLS based reverse proxy in golang </strong>isn’t that hard, and as you can see, it’s worth doing, to learn new things.</p><p>I have hope that you had so much fun reading that as I have writing this project.</p><p>It was a pleasure that you have invested time in this whole blog post. Thanks for that! 🤩</p><p>If you loved what I made, follow me and come and visit (leave a lot of ⭐️, it helps me a lot in keeping the motivation and discipline), take some rest in my tavern 🍻:</p><p><a href="https://github.com/cebilon123/waffle">GitHub - cebilon123/waffle: Web Application Firewall, made in go.</a></p><p>See you next time! 💛</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc9614eae113" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/%EF%B8%8F-%EF%B8%8F-%EF%B8%8F-multi-clients-reverse-proxy-in-golang-bc9614eae113">🚴‍♀️ 🚴‍♀️ 🚴‍♀️ Multi clients reverse proxy in golang.</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating open-source Cloudflare in the golang]]></title>
            <link>https://medium.com/@michalkowal567/creating-open-source-cloudflare-in-the-golang-1b77322eb5f7?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b77322eb5f7</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Mon, 04 Mar 2024 10:54:10 GMT</pubDate>
            <atom:updated>2024-03-04T10:59:04.055Z</atom:updated>
            <content:encoded><![CDATA[<h3>Creating open-source Cloudflare clone in golang</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7yL7QRBUKRlIQkaffMwYuQ.jpeg" /></figure><p>One of the Cloudflare security mechanisms is WAF (Web Application Firewall) which defends app against i.e. DDOS attacks. I was very curious how it’s made so I have starting developing Cloudflare clone in golang and I’m doing it as open-source project.</p><h4>Intro</h4><p>Probabbly everyone heard about Cloduflare. It could be that you used its reverse proxy to defend your website against attacks like DDOS or you have visited some websites and this almost blank page with Cloudflare logo appeard to validate if you are real human, not some kind of bot.<br>Cloudflare is currently the biggest security company in the world, and their solutions and tools inspire other developers that are curious about web security.</p><h4>What’s the Plan?</h4><p>On the beginning I knew that I don’t know anything about writing Web Application Firewall, so I have gathered some information and knowledge about how this thing works. My main source of information was an PDF about how Cloudflare exactly works (the link to this document is available in the readme.md of the GitHub repository).</p><p>First of all: we need to create reverse proxy (that will work as a guard to our services).</p><p>The first layer of security will be addition of TLS protocol to this proxy, in order to cypher the communication between a client and a server. Of course the TLS wouldn’t work without the certificates.</p><blockquote>You can get signed certificate from CA providers like GoDaddy or Lets Encrypt</blockquote><p>We are developing the application on our local environment, so we are going to create local CA and based on that CA we are going to create server.crt (which is the certificate we introduce to the client).</p><p><strong>How TLS works?</strong></p><p>TLS can be treated as SSL (SSL is an ancestor of TLS). The simplified process is basically a TLS Handshake (the communication between client and the server is encrypted or the server is able to require a certificate from the client).</p><h4>Implemntation</h4><p>As mentioned before, the implementation is made in the Golang, repository URL is: <a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a>.</p><p>We need to create abstraction to provide certificates to our server. The abstraction is called <strong>Provider</strong>, and it forces structs to implement two methods: GetTLSCertificate() and GetCACertificatesPool(). The first one will return the server certificate, the second one will return CA certificate that was used to sign the server certificate.</p><pre>package certificate<br><br>import (<br> &quot;crypto/tls&quot;<br> &quot;crypto/x509&quot;<br> &quot;fmt&quot;<br>)<br><br>const (<br> pathCertFile = &quot;.cert/server.crt&quot;<br> pathKeyFile  = &quot;.cert/server.key&quot;<br>)<br><br>// Provider should be implemented by the structs<br>// that are able to provide certificates.<br>type Provider interface {<br> GetTLSCertificate() (*tls.Certificate, error)<br> GetCACertificatesPool() (*x509.CertPool, error)<br>}<br><br>type LocalCertificatesProvider struct {<br> customCaCerts [][]byte<br>}<br><br>func NewLocalCertificatesProvider(caCerts [][]byte) *LocalCertificatesProvider {<br> return &amp;LocalCertificatesProvider{customCaCerts: caCerts}<br>}<br><br>func (l *LocalCertificatesProvider) GetTLSCertificate() (*tls.Certificate, error) {<br> cert, err := tls.LoadX509KeyPair(pathCertFile, pathKeyFile)<br> if err != nil {<br>  return nil, fmt.Errorf(&quot;load x509 key pair: %w&quot;, err)<br> }<br><br> return &amp;cert, nil<br>}<br><br>func (l *LocalCertificatesProvider) GetCACertificatesPool() (*x509.CertPool, error) {<br> caCertPool, err := x509.SystemCertPool()<br> if err != nil {<br>  return nil, fmt.Errorf(&quot;load system ca cert pool: %w&quot;, err)<br> }<br><br> if caCertPool == nil {<br>  caCertPool = x509.NewCertPool()<br> }<br><br> if len(l.customCaCerts) &gt; 0 {<br>  for _, caCert := range l.customCaCerts {<br>   caCertPool.AppendCertsFromPEM(caCert)<br>  }<br> }<br><br> return caCertPool, nil<br>}</pre><p>As you can see, implementation isn’t advenced. Only thing that is worth mentioning are those two paths:</p><ul><li>pathCertFile — this is a path to ther server.crt, certificate for the server signed by our local CA</li><li>pathKeyFile — this is a path to ther server.key, which is basically private key that is uses to encrypt communication from our side</li></ul><p>One more thing: we are having this constructor that takes <strong>caCerts [][]byte </strong>parameter, which is basically used to pass our local CA cert to this provider. When this WAF are going to be production ready, we can then simply pass real world, well known CA cert in the place of our local one.</p><p><strong>Generation of the certificates</strong></p><p>To generate those certificates we are going to use openssl. Basically we need to create CA certificate (certificate authority) that will be used to confirm our server certificate. Then based on the CA we are going to create server certificate. To simplify things I have created makefile:</p><pre>certs_windows:<br> mkdir &quot;.cert&quot;<br> # Create CA (certificate authority)<br> openssl ecparam -out .cert/ca.key -name prime256v1 -genkey<br> openssl req -new -sha256 -key .cert/ca.key -out .cert/ca.csr<br> openssl x509 -req -sha256 -days 3650 -in .cert/ca.csr -signkey .cert/ca.key -out .cert/ca.crt<br><br> # Create server certificate<br> openssl ecparam -out .cert/server.key -name prime256v1 -genkey<br> openssl req -new -sha256 -key .cert/server.key -out .cert/server.csr<br> openssl x509 -req -in .cert/server.csr -CA .cert/ca.crt -CAkey .cert/ca.key -CAcreateserial -out .cert/server.crt -days 3650 -sha256<br> openssl x509 -in .cert/server.crt -text -noout<br><br> # Create cert package in order to access .cert directory<br> (echo package cert &amp; echo &quot;&quot; &amp; echo  &quot;import (&quot; &amp; echo &quot;\&quot;embed\&quot;&quot;  &amp; echo &quot; _ \&quot;embed\&quot;&quot; &amp; echo &quot;)&quot; &amp; echo &quot;&quot;  &amp; echo &quot;//go:embed *&quot; &amp; echo &quot;var Certificates embed.FS&quot;)  &gt; .cert/cert.go</pre><p>As you can see, the make file is being divided into three “parts”.</p><ul><li>First one: creation of the CA related stuff.</li><li>Second one: creation of server related stuff.</li><li>Third: creation of the .go file, that will be used to provide certificates to the application.</li></ul><p>The generated file is simple, and basically uses go embed package to embed certificates.</p><pre>package cert<br><br>import (<br>&quot;embed&quot;<br> _ &quot;embed&quot;<br>)<br><br>//go:embed *<br>var Certificates embed.FS</pre><p>As you can see we are embeding everything from the .cert directory to the Certificates variable which then is used in the main.</p><p><strong>The reverse proxy</strong></p><pre>package proxy<br><br>import (<br> &quot;crypto/rand&quot;<br> &quot;crypto/tls&quot;<br> &quot;errors&quot;<br> &quot;fmt&quot;<br> &quot;log&quot;<br> &quot;net/http&quot;<br> &quot;os&quot;<br> &quot;time&quot;<br><br> &quot;waffle/internal/certificate&quot;<br> &quot;waffle/internal/domain&quot;<br> &quot;waffle/internal/handler&quot;<br>)<br><br>var (<br> ciphers = []uint16{<br>  // TLS 1.3<br>  tls.TLS_AES_128_GCM_SHA256,<br>  tls.TLS_AES_256_GCM_SHA384,<br>  tls.TLS_CHACHA20_POLY1305_SHA256,<br><br>  // ECDSA is about 3 times faster than RSA on the server side.<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // RSA is slower on the server side but still widely used.<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,<br>  tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,<br><br>  // Added so all ciphers are available<br><br>  tls.TLS_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_RSA_WITH_AES_128_GCM_SHA256,<br>  tls.TLS_RSA_WITH_AES_256_GCM_SHA384,<br>  tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,<br>  tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,<br>  tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,<br>  tls.TLS_FALLBACK_SCSV,<br> }<br><br> http11    = &quot;http/1.1&quot;<br> http2     = &quot;h2&quot;<br> http3     = &quot;h3&quot;<br> alpnProto = &quot;acme-tls/1&quot;<br>)<br><br>type Server struct {<br> dns                 domain.NameSystemProvider<br> addr                string<br> certificateProvider certificate.Provider<br>}<br><br>func NewServer(<br> dns domain.NameSystemProvider,<br> addr string,<br> certificateProvider certificate.Provider,<br>) *Server {<br> return &amp;Server{<br>  dns:                 dns,<br>  addr:                addr,<br>  certificateProvider: certificateProvider,<br> }<br>}<br><br>func (s *Server) Start() error {<br> caCertPool, err := s.certificateProvider.GetCACertificatesPool()<br> if err != nil {<br>  return fmt.Errorf(&quot;get ca certificates pool using certificate provider: %w&quot;, err)<br> }<br><br> serverCertificate, err := s.certificateProvider.GetTLSCertificate()<br> if err != nil {<br>  return fmt.Errorf(&quot;get tls certificate using certificate provider: %w&quot;, err)<br> }<br><br> tlsConfig := &amp;tls.Config{<br>  MinVersion:   tls.VersionTLS13,<br>  MaxVersion:   tls.VersionTLS13,<br>  CipherSuites: ciphers,<br>  NextProtos:   []string{http3, http2, http11, alpnProto},<br>  Certificates: []tls.Certificate{*serverCertificate},<br>  ClientAuth:   tls.VerifyClientCertIfGiven,<br>  Rand:         rand.Reader,<br>  RootCAs:      caCertPool,<br>  ClientCAs:    caCertPool,<br> }<br><br> tcpListener, err := tls.Listen(&quot;tcp&quot;, s.addr, tlsConfig)<br><br> router := http.NewServeMux()<br> router.HandleFunc(&quot;/&quot;, handler.RedirectHandler(s.dns))<br><br> server := &amp;http.Server{<br>  Handler:           router,<br>  Addr:              fmt.Sprintf(&quot;%s%s&quot;, &quot;localhost&quot;, s.addr),<br>  ReadHeaderTimeout: 120 * time.Second,<br>  WriteTimeout:      120 * time.Second,<br>  IdleTimeout:       120 * time.Second,<br>  ReadTimeout:       120 * time.Second,<br>  TLSConfig:         tlsConfig,<br>  MaxHeaderBytes:    1048576,<br>  ErrorLog:          log.New(os.Stderr, &quot;&quot;, 0),<br> }<br><br> if err := server.Serve(tcpListener); err != nil &amp;&amp; !errors.Is(err, http.ErrServerClosed) {<br>  return fmt.Errorf(&quot;server serve: %w&quot;, err)<br> }<br><br> return nil<br>}</pre><p>The reverse proxy do few things:</p><ol><li>Fetches the certificates and uses those in the<strong> tls.Config</strong></li><li>Creates server that uses “handler.RedirectHandler(s.dns)”, which basically redirects users to desired destination.</li></ol><p>Later during the development in the place of handler we are going to be able to “decorate” this handler with additional decorates, i.e. rate limiting decorator or ddos prevention decorator.</p><p>Currently we need to add a lot of the stuff to the project, before creating the real defense. We already introduced env reading, using netflix package:</p><pre>package config<br><br>import (<br> &quot;errors&quot;<br> &quot;fmt&quot;<br><br> &quot;github.com/Netflix/go-env&quot;<br>)<br><br>type Environment struct {<br> Database database<br> Extras   env.EnvSet<br>}<br><br>type database struct {<br> URI string `env:&quot;DATABASE_URI&quot;`<br>}<br><br>func (e *Environment) Validate() error {<br> errs := make([]error, 0)<br><br> if len(e.Database.URI) == 0 {<br>  errs = append(errs, fmt.Errorf(&quot;DATABASE_URI is empty&quot;))<br> }<br><br> return errors.Join(errs...)<br>}<br><br>func LoadEnvironmentConfig() (*Environment, error) {<br> var environment Environment<br><br> es, err := env.UnmarshalFromEnviron(&amp;environment)<br> if err != nil {<br>  return nil, fmt.Errorf(&quot;load config from environment variables: %w&quot;, err)<br> }<br><br> environment.Extras = es<br><br> if err := environment.Validate(); err != nil {<br>  return nil, fmt.Errorf(&quot;validate config: %w&quot;, err)<br> }<br><br> return &amp;environment, nil<br>}</pre><p>As you can see, the implementation of the package is really simple, we are basically creating the struct that will be responsible for storing environment variables and then we are adding decorators to the struct field , which then are used to map corresponding values from the Environment Variables.</p><p>The next idea is to introduce YAML config in the place of the database (we came up with the idea, that its better to start simple). We need a way to configure DNS, forwarding and other config.</p><h3>Conclusion</h3><p>The project is in the early stage, but day by day I’m expanding the code base and introduce new functionalities. Of course, this is open source project and I would love to get new contributors that will work with me on this one. If you want to develop your skills and extend the knowledge join!<br>Of course, if you don’t want to develop anything you can allways leave a ⭐️, it doesn’t cost you anything but means a lot for me and keeps me motivated in developing the code.</p><p>Repo: <a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a></p><p>Oh, and I didn’t mention: this is a series, so keep following my medium account if you are curious what happens with the project, but I think it will go to the moon 🚀.</p><p>See you next time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b77322eb5f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[100 commits. Web Application Firewall in GO]]></title>
            <link>https://medium.com/@michalkowal567/100-commits-web-application-firewall-in-go-43e92ddbe71a?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/43e92ddbe71a</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[web-application-firewall]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Sat, 02 Mar 2024 11:18:03 GMT</pubDate>
            <atom:updated>2024-03-02T11:18:03.551Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jTToBp06PBUIiBsh.png" /><figcaption>Image from Cloudflare (<a href="https://www.cloudflare.com/learning/ddos/glossary/web-application-firewall-waf/">https://www.cloudflare.com/learning/ddos/glossary/web-application-firewall-waf/</a>)</figcaption></figure><p>Recently I have started a challenge, called exactly “100 commits” (100 days of commits made every day). To take part in this contest we were forced to come up with an open-source project idea, and as you probably guessed I started with the WAF (Web Application Firewall).</p><h4>Prerequisites</h4><p>For my whole career, I was writing other stuff, basically high-throughput web applications, but lately I was asked if I knew how to write WAF (I didn’t), so it felt natural to go with this project in the contest.</p><p>The main problem with writing a WAF is the lack of information on how to create one (don’t get me wrong, there is a tone, but I mean the exact information on how to write the WAF). I started digging, and I found out few useful sources of information what are the most common attacks, and how to defend the application against those.</p><h4>What is WAF?</h4><p>WAF — Web Application Firewall is a reverse proxy, that checks each incoming request, and in any case, if a given request is suspicious based on different rules it blocks its access.</p><blockquote>Reverse Proxy is a server, that redirects requests to the desired IP address. It works like security guard on the gate to the club, if given person is suspicious it blocks his access to the party, and forces him to leave the queue and find another club.</blockquote><h4>The GO part</h4><p>The project structure is simple and follows the rules of a common golang project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/154/1*E-JBTcgRiR9Qn3B_5o6VzA.png" /></figure><p>In the CMD directory, there is one command for now: proxy. It starts the proxy server that later will guard the access to the desired server.</p><p>The most important part for today is the proxy:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/616a689488e63e86b70f7a54ad27ecd1/href">https://medium.com/media/616a689488e63e86b70f7a54ad27ecd1/href</a></iframe><p>When I started writing the proxy it was a simple HTTP reverse proxy, but later I found out about TLS and especially TLS fingerprints which are very useful in the defending of the web application. I will write more about it in the next post, so keep following because there will be a lot of posts about the project.</p><p>The TLS forces us to create a certificate which can be done by the usage of OpenSSL. I have created a simple makefile, that helps with that</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6577d20f540bcdedb93592c4748b3b4d/href">https://medium.com/media/6577d20f540bcdedb93592c4748b3b4d/href</a></iframe><h4>End for today</h4><p>You can check the repo, star it, and keep following, because oh boy, that&#39;s only day 2.<br><a href="https://github.com/cebilon123/waffle">https://github.com/cebilon123/waffle</a></p><p>I will try to write many posts about the project, those are going to be much longer than this one, it was just written to make you curious about the project and invite you to cooperate and development as this is and will be open source.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43e92ddbe71a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create website for free?]]></title>
            <link>https://medium.com/@michalkowal567/how-to-create-website-for-free-f4cf61bdd48c?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4cf61bdd48c</guid>
            <category><![CDATA[real-estate]]></category>
            <category><![CDATA[food]]></category>
            <category><![CDATA[accounting]]></category>
            <category><![CDATA[website]]></category>
            <category><![CDATA[website-design]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Sun, 12 Nov 2023 20:50:36 GMT</pubDate>
            <atom:updated>2023-11-12T20:50:36.150Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VPioVepm2luxT-JO.jpg" /><figcaption>You are the wizard, that creates his world</figcaption></figure><p>We are living in the times, where you literally don’t need to pay for the creation of the website. I think, that paying once is a huge overkill in terms of price to value ratio. Websites prices, even those made by the freelancers can be huge, and not for people that are starting their business.</p><p>No worries! Currently there is many different options to create those websites for decent price, of course in exchange for your time and a little money (mainly hosting and the domain price).</p><h3><strong>What options do I have?</strong></h3><p>You can use:</p><ul><li><strong>Wordpress</strong></li><li><strong>Wix</strong></li><li><strong>Webflow</strong></li></ul><p>Each of those creates similar but also different approach to the page creation, as well as how those pages are being hosted.</p><p>Let’s talk about each of those. (<strong>If you want to skip this list, I wrote conclusion about those in the paragraph after the webflow</strong>)</p><h4><strong>Wordpress</strong></h4><p>This is the most known tool to create even complex websites with the CMS (content management system, ie. system that let you add new blog posts, products or real estates to your website).</p><p><strong>Pros:</strong></p><ul><li><strong>Really cheap — </strong>While some themes and plugins may have costs, the basic WordPress software is free. This can be cost-effective for individuals and small businesses.</li><li><strong>Easy to use, and create the website — </strong>WordPress is known for its user-friendly interface, making it easy for beginners to create and manage websites without extensive technical knowledge.</li><li><strong>Community support — </strong>Being an open-source platform, WordPress has a large and active community. This means there are plenty of resources, forums, and tutorials available for troubleshooting and learning.</li><li><strong>Content management (CMS) — </strong>Being an open-source platform, WordPress has a large and active community. This means there are plenty of resources, forums, and tutorials available for troubleshooting and learning.</li></ul><p><strong>Cons:</strong></p><ul><li><strong>Security concerns — </strong>Being a widely used platform, WordPress can be a target for hackers. It’s crucial to implement security measures and keep plugins and themes updated</li><li><strong>Customisation Learning Curve — </strong>While the basics are easy to grasp, deep customization might require knowledge of HTML, CSS, and sometimes PHP. Beginners might find certain advanced customizations challenging.</li><li><strong>Performance issues — </strong>Poorly coded themes and plugins or an excessive number of them can lead to slower loading times. It’s essential to optimize for performance.</li><li><strong>Server Resources — </strong>Resource-intensive plugins or poorly optimized themes may require more server resources, potentially leading to higher hosting costs.</li></ul><h4><strong>Wix</strong></h4><p>Wix is a popular website builder that allows users to create websites with ease, even if they don’t have coding skills</p><p><strong>Pros:</strong></p><ul><li><strong>User friendly — </strong>Wix is known for its intuitive drag-and-drop interface, making it easy for beginners to create and customize websites without technical expertise.</li><li><strong>Templates — </strong>Wix offers a wide variety of professionally designed templates, catering to different industries and styles. This allows users to create visually appealing websites quickly.</li><li><strong>Customer support — </strong>Wix provides customer support through various channels, including a knowledge base, tutorials, and a support center, helping users find solutions to common issues.</li><li><strong>Scalability — </strong>Wix is suitable for a range of projects, from small personal blogs to large business websites. Users can easily scale their websites as their needs evolve.</li><li><strong>App market — </strong>Wix has an App Market that provides a range of applications for added functionality, allowing users to integrate features like e-commerce, social media feeds, and more.</li></ul><p><strong>Cons:</strong></p><ul><li><strong>Ad-Supported free plan</strong> — The free plan displays Wix-branded ads on your website. To remove ads and connect a custom domain, users need to upgrade to a premium plan.</li><li><strong>Template limitations </strong>— While Wix offers a variety of templates, some users may find the level of customization limited compared to platforms that allow direct access to code</li><li><strong>Seo limitations </strong>— While Wix has improved its SEO capabilities, some users find it less flexible than other platforms in terms of optimizing for search engines.</li><li><strong>E-commerce transaction fees</strong> — Wix charges transaction fees on e-commerce transactions for users on certain plans, which could impact the overall cost for online businesses.</li></ul><h4>Webflow</h4><p>This is the one that I like the most, it is easy to use, host and add custom domains. Their CMS is really mature, and easy to use, which makes it really cool tool for accountant, real estate, restaurant where you need modify content often.</p><p><strong>Pros:</strong></p><ul><li><strong>Visual Design Interface </strong>— Webflow’s visual design interface allows users to create websites visually, making it easy for designers to bring their ideas to life without extensive coding knowledge.</li><li><strong>Interactions and Animations</strong> — Webflow offers advanced interaction and animation capabilities, allowing users to create dynamic and engaging websites.</li><li><strong>CMS Capabilities</strong> — Similar to WordPress, Webflow provides content management system (CMS) features, enabling users to easily add and manage content on their websites.</li><li><strong>Responsive Design</strong> — Webflow facilitates the creation of responsive websites that adapt to various screen sizes, ensuring a seamless user experience across devices.</li><li><strong>Collaboration Tools </strong>— Webflow provides collaboration features, allowing multiple team members to work on a project simultaneously.</li></ul><p><strong>Cons:</strong></p><ul><li><strong>Pricing </strong>— Webflow’s pricing structure may be considered higher compared to some other website builders, especially for those with budget constraints.</li><li><strong>Learning Curve </strong>— While Webflow is user-friendly, mastering its advanced features may require some time and effort, particularly for beginners.</li><li><strong>Limited Third-Party Integrations</strong> — Webflow has fewer third-party integrations compared to some other platforms, which might limit the ability to connect with certain tools and services.</li><li><strong>Custom Code Limitations </strong>— Users with advanced coding needs might find certain limitations in terms of adding custom code compared to platforms with more open architectures.</li></ul><h3><strong>Conclusion about builders</strong></h3><p>Those builders are really amazing, and if you have a lot of time, you can save a lot of money. If you want to assign domain to those websites, with the help of youtube and google you can really build your own website for decent price.</p><p>Trust me, you will need to spend a lot of time on that, firstly to learn those tools, then to get even more knowledge about hosting and setting the domain on those tools.</p><h3>Website for free, made by professional</h3><p>You don’t need to pay for the website creation, and it will be delivered quickly, in the quality that is worth it.</p><p>This is not a bait, this is real thing, that I’m trying to do.</p><p>How it works?<br>Basically you need to contact me, using my private mail:</p><p>➡️ michal@kowaldev.com</p><p>I will respond to you as fast as I can 💪.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4cf61bdd48c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How can I automate fetching files from mail’s attachments without coding?]]></title>
            <link>https://medium.com/@michalkowal567/how-can-i-automate-fetching-files-from-mails-attachments-without-coding-72dd7d58e284?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/72dd7d58e284</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[accounting]]></category>
            <category><![CDATA[real-estate]]></category>
            <category><![CDATA[automation]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Thu, 09 Nov 2023 18:06:26 GMT</pubDate>
            <atom:updated>2023-11-09T18:06:26.461Z</atom:updated>
            <content:encoded><![CDATA[<p>How can I automate fetching files from mail’s attachments, without coding?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*bwpIoiLGab5_FynZ.jpg" /><figcaption>Me, thinking what can I automate</figcaption></figure><p>There is a trend that lately has blown up, that’s automation, and I didn’t know what to automate, until now. This article is for all of you there, that spend time on the repetitive tasks, or on those that takes a lot of the time, which in reality could be automated.</p><h3>Invoices</h3><p>As the owner of the company, I’m getting a lot of the invoices, all those on multiple emails, which then needs to be checked by me, multiple times per week, and trust me, even if it takes 2 mins to handle one invoice, it takes much more to handle 40/50 monthly.</p><p>The goal was simple: <strong>to spend less time on those invoices</strong></p><p>Of course, as software developer I could write some code to automate this stuff easily, but I have chosen something else: <strong>Zapier.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*CT9VUFP7I_hirVtp.png" /><figcaption>The zapier logo</figcaption></figure><p><strong>Zapier</strong>, the unsung hero of automation, became the cornerstone of my plan to liberate myself from the invoice labyrinth. The goal was clear: spend less time on the administrative hassle and redirect that energy towards more meaningful endeavors.</p><p>Here’s the game plan: Zapier would seamlessly fetch invoices from my emails, neatly organize them on Google Drive, and like clockwork, shoot them off to my accountant on the last day of the month. A few clicks, a bit of configuration, and voila — a well-oiled machine ready to tackle the invoice onslaught.</p><p>But why Zapier? Because it’s user-friendly, versatile, and most importantly, it gets the job done without the need for an advanced degree in coding. As the founder of an email automation agency, I wanted a solution that not only streamlined my own processes but also showcased the power of automation to potential clients.</p><p>Picture this: Your inbox, a serene oasis cleared of the invoice clutter, your Google Drive, a neatly organized library of financial documentation, and your accountant, receiving a perfectly compiled monthly report without you lifting a finger.</p><p>This isn’t just about saving time; it’s about reclaiming your sanity and empowering your business to thrive. So, if you’re ready to break free from the chains of manual invoice handling, consider the wonders of automation. Zapier might just be the magic wand your business needs.</p><p><strong>The safety</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/0*N1Na2TatmtNWeH2G.jpg" /></figure><p>Now, some skeptics might raise an eyebrow and ask, “But what about the safety of automating sensitive tasks like handling invoices?” Valid concern. Security should never take a backseat, especially when it comes to financial matters.</p><p>Zapier, with its array of automation apps, isn’t just a time-saver; it’s a guardian of your digital fortress. Imagine this: invoices flooding in from various emails, each potentially harboring a digital intruder. With Zapier’s automation apps, the process becomes a security checkpoint, not a vulnerability.</p><p>Here’s the secret sauce: by incorporating Zapier apps that include robust security measures, you’re not just automating; you’re fortifying your defenses. The automation process becomes a meticulous inspector, scanning each invoice for hidden threats.</p><p>Let’s break it down. Zapier allows you to integrate apps that can scrutinize attachments, ensuring they’re free from malicious software or viruses. Before an invoice even reaches your meticulously organized Google Drive, it undergoes a security clearance, courtesy of your chosen automation app.</p><p>This isn’t just about efficiency; it’s about safeguarding your business from potential digital hazards. As the founder of an email automation agency, security is paramount for both my business and my clients’. Zapier’s security-focused automation apps add an extra layer of confidence in the automated process.</p><p>So, if the safety of your financial data keeps you up at night, consider Zapier not just as an efficiency tool but as a digital security guard, diligently patrolling the gates of your virtual domain. With the right automation apps in place, you can rest easy, knowing that every invoice that enters your system is not just a number; it’s a trusted guest.</p><p><strong>Conclusion</strong></p><p>In the dynamic world of business, where time is a precious currency and efficiency is the name of the game, automation emerges as the unsung hero. This article has chronicled the transformative journey from the clutches of manual invoice handling to the freedom of streamlined processes, with Zapier leading the charge.</p><p>Zapier, with its user-friendly interface and security-focused automation apps, has proven to be more than just a tool; it’s a strategic ally in the pursuit of business excellence. From inbox chaos to an organized financial haven on Google Drive, the power of automation is not just about saving time; it’s about redirecting that time towards meaningful endeavors.</p><p>As the founder of an email automation agency, I’ve embraced automation not just as a solution for my own business but as a beacon for others seeking efficiency and innovation. <strong>If you find yourself resonating with the challenges of manual tasks or have questions about implementing automation in your workflow, don’t hesitate to comment</strong>. I’m here to help you navigate the exciting terrain of automation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72dd7d58e284" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a Dead-Simple and Reusable CQRS Golang Package]]></title>
            <link>https://medium.com/@michalkowal567/creating-a-dead-simple-and-reusable-cqrs-golang-package-f48ade506b86?source=rss-59485644f376------2</link>
            <guid isPermaLink="false">https://medium.com/p/f48ade506b86</guid>
            <category><![CDATA[eventstorming]]></category>
            <category><![CDATA[cqrs]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[ddd]]></category>
            <dc:creator><![CDATA[Michalkowal]]></dc:creator>
            <pubDate>Thu, 13 Apr 2023 21:15:40 GMT</pubDate>
            <atom:updated>2023-04-13T21:15:40.029Z</atom:updated>
            <content:encoded><![CDATA[<h3>How I created the CQRS package in Golang?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*1UdYnye8wEnTLnEhAcmrfg.png" /></figure><h3>What is the CQRS?</h3><p>CQRS stands for Command Query Responsibility Segregation. It is a pattern that separates read and write models in your application. It is a good practice to separate read and write models. It is also handy for the <strong>DDD </strong>(Domain Driven Design) approach.</p><h3>Package implementation</h3><p>The package is divided into four main parts:</p><ol><li>Command Part: This section contains the command-related code, including the Command struct and the CommandHandlerFunc. We use a map to store command handlers and a mutex for thread safety.</li><li>Payload Part: This part includes the TryMapPayload function, which attempts to map the payload to a specified type. It returns an error if the payload is nil or not of the given type.</li><li>Query Part: This section contains the query-related code, including the Query struct and the QueryHandlerFunc. Similar to the command part, we use a map to store query handlers and a mutex for thread safety.</li><li>Result Mapping: The tryMapResult function helps map the result of a query to the desired type, returning an error if the result is nil or not of the specified type.</li></ol><h3>My motivation</h3><p>Simply: I couldn’t find any nice, easy-to-adapt CQRS package to golang.</p><p>I’m building a new project using <em>Event Storming</em> for the analytics of business processes and to build meaningful domain objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*1UdYnye8wEnTLnEhAcmrfg.png" /></figure><p>Meaning of each sticky note:</p><ul><li>blue: command</li><li>yellow: aggregate</li><li>orange: domain event</li><li>purple: policy (when X happens do Y)</li></ul><p>As you can see, whenever there is an action made on the “Invitation” aggregate, we are calling the <strong>command </strong>which produces the domain event that can be consumed, serialized (i.e. as the event sourcing event), or whatever can be made with such a thing.</p><p>The keyword <strong>“command”. </strong>Of course, it is being called on the aggregate which is not necessarily a CQRS thing, but it is kind of related. Basically, we are having bounded context for a given domain abstraction, and context actions (commands) can be called separately. In an ideal scenario, each CQRS command calls one domain command on the aggregate and that&#39;s part of the reason that I needed the CQRS here.</p><p>The second reason is basically, that projects using such an approach are really scalable.</p><h3>The plan</h3><p>The idea was simple: we need a command and query dispatcher (where a query can return results). Also, the registration of each command and query handler should be dead simple.</p><p>For a very long time, I was coding in C#, and I was trying to reproduce the approach with generics, but Go generics are a little bit different than those in C#.</p><h3>The implementation</h3><p>With the idea of making it simple, I came up with package level based approach, which indeed is pretty simple.</p><p>The whole idea was to implement command handlers that can be easily registered and used on the package level so the API could look like this:</p><pre>const (<br> commandKey = &quot;commandKey&quot;<br>)<br><br>type CommandTestPayload struct {<br> FirstName string<br>}<br><br>func main() {<br> cqrs.RegisterCommandHandlerFuncs(commandKey, func(ctx context.Context, cmd cqrs.Command) error {<br>  // TryMapPayload is a helper function that tries to map the payload to the given type. It returns an error<br>  //if the payload is nil or if the payload is not of the given type.<br>  payload, err := cqrs.TryMapPayload[CommandTestPayload](cmd.Payload)<br>  if err != nil {<br>   return err<br>  }<br><br>  println(payload.FirstName)<br><br>  return nil<br> })<br><br><br> ctx := context.Background()<br> err := cqrs.DispatchCommand(ctx, cqrs.Command{<br>  Key: commandKey,<br>  Payload: CommandTestPayload{<br>   FirstName: &quot;John&quot;,<br>  },<br> })<br> if err != nil {<br>  log.Print(err)<br> }<br>}</pre><p>This is basically the implementation of the CQRS’s commands part:</p><pre>// Package cqrs provides a simple implementation of the command query responsibility segregation pattern.<br>package cqrs<br><br>import (<br> &quot;context&quot;<br> &quot;errors&quot;<br> &quot;fmt&quot;<br> &quot;sync&quot;<br>)<br><br>// Payload is the command payload. It can be any type.<br>// It can be then deserialized to the desired type using the TryMapPayload function.<br>type Payload any<br><br>// Command is the command that is dispatched to the command handler.<br>type Command struct {<br> Key     Key     // Key is the unique identifier of the command.<br> Payload Payload // Payload is the command payload.<br>}<br><br>// CommandHandlerFunc is the function that handles the command.<br>type CommandHandlerFunc func(ctx context.Context, cmd Command) error<br><br>var (<br> // commandHandlers is a map of command keys to a slice of command handler functions.<br> commandHandlers = make(map[Key][]CommandHandlerFunc)<br> commandMutex    sync.Mutex<br>)<br><br>// RegisterCommandHandlerFuncs registers the given command handler functions for the given command key.<br>func RegisterCommandHandlerFuncs(cmdKey Key, h ...CommandHandlerFunc) {<br> commandMutex.Lock()<br> defer commandMutex.Unlock()<br><br> commandHandlers[cmdKey] = append(commandHandlers[cmdKey], h...)<br>}<br><br>var ErrCommandHandlerFuncNotFound = errors.New(&quot;command handler func for given command not found&quot;)<br><br>// DispatchCommand dispatches the given command to the command handler.<br>func DispatchCommand(ctx context.Context, cmd Command) error {<br> v, ok := commandHandlers[cmd.Key]<br> if !ok {<br>  return fmt.Errorf(&quot;%w; %s&quot;, ErrCommandHandlerFuncNotFound, cmd.Key)<br> }<br><br> for _, h := range v {<br>  if err := h(ctx, cmd); err != nil {<br>   return err<br>  }<br> }<br><br> return nil<br>}</pre><p>As you can see, the implementation is indeed dead simple. We are just having a few types that are wrapping base or any type from golang and one struct.</p><p>As I have written before, golang’s generics are a little bit different (and annoying) compared to C# or Java, so I needed to find an approach to easily register command handlers for given commands. With the “dead simple” approach in mind, I have created the type:</p><pre>type Key string</pre><p>Which basically is used to create a relation one to many between command and command handlers for a given command type. The key is basically the string type, it gives some freedom in naming those commands and queries.</p><p>The RegisterCommandHandlerFuncs function basically registers command handlers for a given key:</p><pre>func RegisterCommandHandlerFuncs(cmdKey Key, h ...CommandHandlerFunc) {<br> commandMutex.Lock()<br> defer commandMutex.Unlock()<br><br> commandHandlers[cmdKey] = append(commandHandlers[cmdKey], h...)<br>}</pre><p>With the public package in mind, I made it so it could be registered in any place of code. I know, that I gave free hand to consumers of the package, but this is the key: to reuse it whenever you need the CQRS.</p><p>Then there is a Dispatch function:</p><pre>func DispatchCommand(ctx context.Context, cmd Command) error {<br> v, ok := commandHandlers[cmd.Key]<br> if !ok {<br>  return fmt.Errorf(&quot;%w; %s&quot;, ErrCommandHandlerFuncNotFound, cmd.Key)<br> }<br><br> for _, h := range v {<br>  if err := h(ctx, cmd); err != nil {<br>   return err<br>  }<br> }<br><br> return nil<br>}</pre><p>Which basically iterates through each handler and calls it, if there would be any errors in any handler for given command it would return it to the caller.</p><h3>Conclusion</h3><p>This dead-simple and reusable CQRS Golang package allows you to implement the CQRS pattern with minimal complexity. By focusing on reusability and simplicity, the package enables developers to quickly integrate the pattern into their projects and spend more time on their application’s core functionality. Give this package a try and see how it can improve your development experience!</p><p>The package:<br><a href="https://github.com/cebilon123/cqrs">https://github.com/cebilon123/cqrs</a></p><p>Contribute and star! 😃</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f48ade506b86" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>