<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Felipe Girotti on Medium]]></title>
        <description><![CDATA[Stories by Felipe Girotti on Medium]]></description>
        <link>https://medium.com/@felipegirotti?source=rss-9b1de7721140------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dqv8bpZYnDRFKf2QjFqXGg.jpeg</url>
            <title>Stories by Felipe Girotti on Medium</title>
            <link>https://medium.com/@felipegirotti?source=rss-9b1de7721140------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:06:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@felipegirotti/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Solve error apt-get update on the docker container Debian on Mac M1]]></title>
            <link>https://medium.com/@felipegirotti/solve-error-apt-get-update-on-the-container-debian-on-mac-m1-7369698049a1?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/7369698049a1</guid>
            <category><![CDATA[debian]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[containers]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Tue, 18 Apr 2023 18:15:43 GMT</pubDate>
            <atom:updated>2023-04-18T18:16:56.621Z</atom:updated>
            <content:encoded><![CDATA[<p>When you ran the apt-get update inside a container, or build a container and got this error or similar:</p><pre>Err:1 http://deb.debian.org/debian bullseye InRelease</pre><p>You should disable the iCloud Private Relay</p><p>Todo that goes to “<strong>Settings</strong> -&gt; <strong>Apple ID</strong>” and turn off.</p><p>And that it!</p><p>Reference:</p><p><a href="https://github.com/docker/for-mac/issues/6030#issuecomment-992590521">https://github.com/docker/for-mac/issues/6030#issuecomment-992590521</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7369698049a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Software Developer Unit Test]]></title>
            <link>https://medium.com/@felipegirotti/the-software-developer-unit-test-eb694a458820?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb694a458820</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Sun, 05 Feb 2023 18:57:17 GMT</pubDate>
            <atom:updated>2023-02-05T18:57:17.967Z</atom:updated>
            <content:encoded><![CDATA[<p>An analogy between tests and software development.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*82sSAXpHUw7NWWf-KjtD_Q.png" /></figure><p>In the world of software development, unit tests are practically mandatory to maintain a healthy codebase, and the more test coverage (how much of your code is being used by your unit tests), the better and healthier your code is (but it’s not that simple).</p><p>Throughout my career, I’ve seen and experienced many software development experiences (web-focused), and people always fall into the myth that having high coverage is synonymous with developing good, maintainable, and healthy code.</p><blockquote>Having at least 80% test coverage, or extreme levels of 100%, guarantees that we are bug-free!</blockquote><p>Indeed, when we have high coverage, the code tends to be better because to make a testable code, we have to follow some patterns to be able to perform tests and possible mocks, such as using the <a href="https://en.wikipedia.org/wiki/Dependency_inversion_principle"><strong>Dependency Inversion</strong></a> principle, as well as helping to better understand how that component works, thus providing a good complementary documentation base.</p><h4>Why unit tests are not enough</h4><p>But the unit test only covers a small unit of your component without the context around it. Just the method of a class that inputs data, processes (searches in some storage/API), and returns the result.</p><p>This leaves some very large context gaps, which when executed together with other components (remember software is composed of several interconnected components) simply break.</p><p>They break because they don’t know the context, they didn’t expect a scenario where the data is different, or where the actual storage (Database, S3, Filesystem) or API responded differently.</p><p>And this is usually only captured in production, by the application user, or by the QA team, and this becomes a big problem to be solved at that moment, wasting the team’s time to find the root of the problem, what the unexpected scenario was, to make the changes in the code that produce the expected result.</p><p>All of this causes a great deal of stress for the team, as it is like acting as firefighters, putting out fire outbreaks all the time.</p><p>To mitigate this problem, I<a href="https://en.wikipedia.org/wiki/Integration_testing"><strong>ntegration tests or feature</strong></a> tests and <a href="https://www.cypress.io/how-it-works/"><strong>end-to-end</strong></a> tests should be done, as suggested in <a href="https://martinfowler.com/articles/practical-test-pyramid.html"><strong>Martin Fowler’s testing pyramid</strong></a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*IBpGR7l_cU-NlkIBSr81AQ.png" /><figcaption><a href="https://martinfowler.com/bliki/TestPyramid.html">https://martinfowler.com/bliki/TestPyramid.html</a></figcaption></figure><p>Going deeper, I always suggest that <strong>the integration test has more weight than the unit test </strong>so that we cover the real scenarios and real possibilities.</p><p>An example is a service that calls a repository and performs a complex database query to return some data. When doing the integration test, you query the database and test if your complex query is functioning, while in the unit test, we only make the mock without testing if the query is actually working as it should.</p><h4>The Software Engineer Without Context</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*8xhzAsp0FaSDMjlCEwRa_g.png" /></figure><p>In modern software development, we usually work in multi-disciplinary teams, where we have a product person (PM/PO), a UI/UX person, DevOps, Developers, and in some cases, people from other specific areas. With more people with various expertise, we have a big picture and the product evolves better.</p><p>Within this scenario, what is usually done, the product person always seeks improvement, so they hold a large part of the context of the entire product, how it works, what are the main customer profiles, what are flows should be followed, etc.</p><p>When there is some new functionality to be developed, it is passed to the team, the PM/PO together with UI/UX does all the discovery of how this works for the users, and in conjunction with the developers to know if there is technical feasibility.</p><p>When passed to the developer to implement the functionality, it is done as implicit that he knows the context, often this functionality belongs to a larger scope, an improvement in flow, a new integration, a new functionality within something already existing.</p><p>In this scenario, the developer is without the context, and what I perceive over time is that many do not care about the context, only do what is written in the Jira ticket, without caring for the existing Big Picture.</p><h4>The Software Developer Unit Test</h4><p>They just do, a great executor, often without understanding, without questioning, and without context.</p><p>When they finish their task, they just move it to complete, saying that they have unit tests, that coverage is good. But in reality, it doesn’t work as it should, because there is all the background context that changes the expected scenario.</p><p>Many of these developers (unit test) expect the PM/PO to describe all possible scenarios, and only the Tech-Lead (Staff, Principal, Architect) are the people who should understand the entire context and write in the ticket what should be done technically. In my view, the entire team has the responsibility, but it is the developer who is involved in the functionality who should have a sense of responsibility, not just deliver what is written in the ticket.</p><p>In many scenarios, the PM/PO has a total focus on the product as a whole, and it is the software engineering team’s role to help them understand how it is done within the application and how this can affect the overall understanding of the functionality. This can completely change the way it will be developed. For example, we can skip a step because it is already implemented in another part of the system, or we have to do something first, to prepare the data, etc, etc…</p><p>Therefore, don’t be a unit test developer, always try to understand the context to the fullest, and don’t think that this is just the PM/PO’s role, understanding the business, the context, how things work in the big picture is fundamental for your professional growth and fundamental for product improvement.</p><p>Clear communication is essential, don’t be afraid to express yourself verbally, to not understand the context, and always have a total understanding before writing the first line of code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb694a458820" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[O Desenvolvedor Teste Unitário]]></title>
            <link>https://medium.com/@felipegirotti/o-desenvolvedor-teste-unit%C3%A1rio-871e7972dad8?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/871e7972dad8</guid>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[test]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Tue, 27 Dec 2022 13:11:59 GMT</pubDate>
            <atom:updated>2022-12-27T19:06:47.598Z</atom:updated>
            <content:encoded><![CDATA[<p>Uma analogia entre testes e desenvolvedor de software</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8_vcMPdQOiGONG-AjkOEqw.png" /></figure><p>No mundo de desenvolvimento de software testes unitários são praticamente obrigatório para manter uma base de código sadia, e quando mais cobertura de testes (quanto do seu código está sendo usado pelos seus testes unitários), melhor e mais saudável é seu código (mas não é bem assim).</p><p>Ao longo de minha carreira já vi e vivi muitas experiencias em desenvolvimento de software (voltado a web), e sempre pessoas caem no mito de que ter uma cobertura alto é sinônimo de que estamos desenvolvendo um código bom, manutenível e saudável.</p><blockquote>Ter no mínino 80% de cobertura de testes, ou níveis extremos de 100% garantem que estamos livres de bugs!</blockquote><p>De fato quando temos uma cobertura grande, usualmente o código tende a ser melhor, pois para fazer um código testável, temos que seguir alguns padrões para poder efetuar os testes e os mocks possíveis, como utilizar o principio de <a href="https://pt.wikipedia.org/wiki/Princ%C3%ADpio_da_invers%C3%A3o_de_depend%C3%AAncia"><strong>Dependency Injection</strong></a>, além de ajudar a entender melhor como funciona aquele componente, sendo assim uma boa base de documentação complementar.</p><h3>Porque testes unitários não são suficientes</h3><p>Mas o teste unitário, ele cobre apenas a pequena unidade de seu componente sem o contexto em volta dele. Apenas o método de uma classe que imputa dados, processa (busca em algum storage/api) e devolve o resultado.</p><p>Isto deixa algumas brechas de contexto bem grandes, que quando executamos em conjunto com outros componentes (lembre-se o software é composto de vários componentes interligados) eles simplesmente quebram.</p><p>Quebram porque não sabem do contexto, não esperavam um cenário onde os dados são diferentes, ou onde o real storage (Database, S3, Filesystem) ou API respondessem de forma diferente.</p><p>E normalmente isto só é capturado em produção, pelo usuário da aplicação, ou pela equipe de QA, e isto torna-se um grande problema para ser resolvido naquele momento, gastando tempo da equipe para descobrir a raiz do problema, qual o cenário inesperado, para efetuar as mudanças no código que produzam o resultado esperado.</p><p>Tudo isto causa na equipe uma grande carga de estresse, pois é como atuar como bombeiros, a todo tempo apagando focos de incêndio.</p><p>Para mitigar este problema deve ser feito os testes de <a href="https://pt.wikipedia.org/wiki/Teste_de_integra%C3%A7%C3%A3o"><strong>Integração ou Feature</strong></a> mais os testes <a href="https://www.cypress.io/how-it-works"><strong>End two End</strong></a>, como sugerido na <a href="https://martinfowler.com/articles/practical-test-pyramid.html"><strong>pirâmide de testes de Martin Fowler</strong></a><strong>.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*ho3TNCRaLEAVYyiFsSXOLA.jpeg" /></figure><p>Mas indo mais a fundo, eu sempre sugiro que <strong>o teste de integração tenha mais peso do que o unitário</strong>, pois assim cobrimos os reais cenários e possibilidades reais.</p><p>Um exemplo é um service que chama um repository e faz uma consulta complexa no banco de dados, para retornar alguns dados. Quando se está fazendo o teste de integração, você busca no banco de dados, e testa se sua consulta complexa está em funcionamento, já no teste unitário apenas fazemos o mock sem testar se a consulta de fato está funcionando como deveria.</p><h3>O Desenvolvedor sem contexto</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*8xhzAsp0FaSDMjlCEwRa_g.png" /></figure><p>No desenvolvimento moderno de software normalmente trabalhamos em equipes multi-diciplinares, onde temos uma pessoa de produto (PM/PO) uma pessoa de UI/UX, DevOps, Desenvolvedores e em alguns casos, pessoas de outras areas especificas. Assim com mais pessoas em diversas espertises temos uma big picture e o produto evolui melhor.</p><p>Dentro deste cenário o que normalmente é feito, a pessoa de produto sempre busca a melhora, então detém grande parte do contexto de todo o produto, como funciona, quais são os principais perfis dos clientes, quais os fluxos que deve seguir, etc.</p><p>Quando se tem alguma nova funcionalidade a ser desenvolvida é passado para a equipe, o PM/PO junto com UI/UX faz toda a descoberta de como isto funciona para os usuários, e em conjunto com os desenvolvedores para saber se tem a viabilidade técnica.</p><p>Quando se passa para o desenvolvedor implementar a funcionalidade se faz como implícito de que ele saiba do contexto dela, muitas vezes esta funcionalidade pertence a um escopo maior, uma melhoria de fluxo, uma nova integração, uma nova funcionalidade dentro de algo já existente.</p><p>Neste cenário é que o desenvolvedor fica sem o contexto, e o que percebo ao longo do tempo é que muitos não se importam com o contexto, apenas faz o que está escrito no ticket do Jira, sem se importar para a Big Picture existente.</p><h3>O Desenvolvedor Teste Unitário</h3><p>Ele apenas faz, um grande executor, muitas vezes sem entender, sem questionar, e sem contexto.</p><p>Quando termina sua task ele apenas move para concluído, dizendo que tem testes unitários, que a cobertura está boa. Mas na realidade não funciona como deveria, pois tem todo o contexto atrás que altera o cenário previsto.</p><p>Muitos destes desenvolvedores (teste unitário) esperam que o PM/PO descrevam todos os cenários possíveis, de que apenas o Tech-Lead (Staff, Principal, Architect) são as pessoas que devem entender todo o contexto e escrever no ticket o que deve ser feito tecnicamente. <br>Na minha visão, o time como todo tem a responsabilidade, mas é o desenvolvedor que está envolvido na funcionalidade deve ter a noção de responsabilidade, não apenas de entregar o que está escrito no ticket.</p><p>Em vários cenários o PM/PO tem o total foco no produto em geral, e é papel do time de engenharia de software ajudar a ele entender como é feito dentro da aplicação, e como isto pode afetar o entendimento geral da funcionalidade.<br>Isto pode mudar totalmente a forma de como será desenvolvido. Exemplo, podemos pular uma etapa pois já é implementado em outra parte do sistema, ou temos que fazer algo antes, para preparar os dados, etc, etc…</p><p>Por isto não seja um desenvolvedor teste unitário, sempre tente entender o contexto ao máximo, não pense que isto é apenas o papel do PM/PO, entender o negocio, o contexto, como as coisas funcionam no big picture é fundamental para o seu crescimento profissional, e fundamental para a melhoria do produto.</p><p>A clareza na comunicação é fundamental, não tenha medo de se expressar verbalmente, de não entender o contexto, sempre tenha o entendimento total antes de efetuar a primeira linha de código.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=871e7972dad8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes namespace stuck on terminating status]]></title>
            <link>https://medium.com/@felipegirotti/kubernetes-namespace-stuck-on-terminating-status-c79712a1fea5?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/c79712a1fea5</guid>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Wed, 03 Aug 2022 17:13:05 GMT</pubDate>
            <atom:updated>2023-05-09T19:45:31.569Z</atom:updated>
            <content:encoded><![CDATA[<p>Sometimes when we delete the namespace at Kubernetes this takes much time and is never deleted.</p><p>This happens due to the Kubernetes finalizers, it checks if have some resources pending to delete at the namespace to delete per complete the namespace.</p><p>To check what&#39;s going on at the status of the namespace you can check</p><pre>kubectl get ns {NAMESPACE} -o yaml</pre><p>In this case in my cluster GKE all pending delete namespaces are due this:</p><pre>status:<br>  conditions:<br>  - lastTransitionTime: &quot;2022-07-28T11:59:45Z&quot;<br>    message: &#39;Discovery failed for some groups, 1 failing: unable to retrieve the<br>      complete list of server APIs: <strong>packages.operators.coreos.com/v1</strong>: the server is<br>      currently unable to handle the request&#39;<br>    reason: DiscoveryFailed<br>    status: &quot;True&quot;</pre><p>The API <strong>packages.operators.coreos.com/v1.<br></strong>I found this issue on Github related to this: <a href="https://github.com/operator-framework/operator-lifecycle-manager/issues/2001">https://github.com/operator-framework/operator-lifecycle-manager/issues/2001</a></p><p>And when I did check the resource:</p><pre>kubectl get apiservices.apiregistration.k8s.io </pre><pre>NAME                                   SERVICE                      AVAILABLE                 AGE</pre><pre>v1.packages.operators.coreos.com       olm/packageserver-service    False (<strong>ServiceNotFound</strong>)   23d</pre><p>So I just deleted that and then all pending termination was gone!</p><pre>kubectl delete apiservices.apiregistration.k8s.io v1.packages.operators.coreos.com</pre><h4>Force delete namespace</h4><p>When you have other scenarios for stuck termination I use this snipped collected from S<a href="https://stackoverflow.com/questions/52369247/namespace-stuck-as-terminating-how-i-removed-it#answer-53661717">tackOverflow</a></p><pre>NAMESPACE={YOUR_NAMESPACE_TO_DELETE}</pre><pre>kubectl proxy &amp;</pre><pre>kubectl get namespace $NAMESPACE -o json |jq &#39;.spec = {&quot;finalizers&quot;:[]}&#39; &gt;temp.json</pre><pre>curl -k -H &quot;Content-Type: application/json&quot; -X PUT --data-binary @temp.json 127.0.0.1:8001/api/v1/namespaces/$NAMESPACE/finalize</pre><p>Or if you need to delete a bunch of namespace stuck on Termination, the simple script</p><pre>for p in $(kubectl get pods | grep Terminating | awk &#39;{print $1}&#39;); do kubectl patch pod $p -p &#39;{&quot;metadata&quot;:{&quot;finalizers&quot;:null}}&#39; ;done</pre><h4>Resources:</h4><p><a href="https://www.redhat.com/sysadmin/troubleshooting-terminating-namespaces">How to fix Kubernetes namespaces stuck in the terminating state</a></p><p><a href="https://www.ibm.com/docs/en/cloud-private/3.2.0?topic=console-namespace-is-stuck-in-terminating-state">https://www.ibm.com/docs/en/cloud-private/3.2.0?topic=console-namespace-is-stuck-in-terminating-state</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c79712a1fea5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mysql Select Empty JSON Array]]></title>
            <link>https://medium.com/@felipegirotti/mysql-select-empty-json-array-53c1194e5efa?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/53c1194e5efa</guid>
            <category><![CDATA[json-mysql]]></category>
            <category><![CDATA[json]]></category>
            <category><![CDATA[mysql-8]]></category>
            <category><![CDATA[mysql]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Sun, 24 Apr 2022 20:45:37 GMT</pubDate>
            <atom:updated>2022-04-24T20:46:33.786Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/557/1*ixp7gBDyghTtKqbPKjYKLw.png" /></figure><p>Mysql introduces many features to work and manipulate JSON since version 5.7.8.</p><p>We will simulate the options to search empty arrays. <br>For that, we use MYSQL 8.</p><p>We have this table and data below:</p><pre>CREATE TABLE `courses` (<br>  `id` int NOT NULL AUTO_INCREMENT,<br>  `data` json NOT NULL,<br>  PRIMARY KEY (`id`) <br>) ENGINE=InnoDB;</pre><pre>INSERT INTO `courses` (`data`) VALUES <br>(&#39;{&quot;name&quot;: &quot;Medicine&quot;, &quot;codes&quot;: [&quot;3064&quot;, &quot;3065&quot;]}&#39;),<br>(&#39;{&quot;name&quot;: &quot;Dentistry&quot;, &quot;codes&quot;: [&quot;3070&quot;]}&#39;),<br>(&#39;{&quot;name&quot;: &quot;Pharmacology&quot;, &quot;codes&quot;: []}&#39;),<br>(&#39;{&quot;name&quot;: &quot;Toxicology&quot;, &quot;codes&quot;: []}&#39;);</pre><h3>JSON_EXTRACT</h3><pre>SELECT data FROM courses WHERE JSON_EXTRACT(data, &#39;$.codes&#39;) = &#39;[]&#39;;</pre><p>This supposes to return the Pharmacology and Toxicology courses, in fact, return empty results.</p><p>There is a trick to using <strong>JSON_EXTRACT</strong> to cast the comparable the object <strong>JSON_ARRAY</strong></p><pre>SELECT data FROM courses WHERE JSON_EXTRACT(data, &#39;$.codes&#39;) = JSON_ARRAY();</pre><p>Bingo! The result was as expected!</p><pre>+---------------------------------------+<br>| data                                  |<br>+---------------------------------------+<br>| {&quot;name&quot;: &quot;Pharmacology&quot;, &quot;codes&quot;: []} |<br>| {&quot;name&quot;: &quot;Toxicology&quot;, &quot;codes&quot;: []}   |<br>+---------------------------------------+<br>2 rows in set (0.01 sec)</pre><h3><strong>JSON_LENGTH</strong></h3><pre>SELECT data FROM courses WHERE JSON_LENGTH(data, &#39;$.codes&#39;) = 0;</pre><p>The result as expected, only return records with empty codes</p><pre>+---------------------------------------+<br>| data                                  |<br>+---------------------------------------+<br>| {&quot;name&quot;: &quot;Pharmacology&quot;, &quot;codes&quot;: []} |<br>| {&quot;name&quot;: &quot;Toxicology&quot;, &quot;codes&quot;: []}   |<br>+---------------------------------------+<br>2 rows in set (0.01 sec)</pre><h3>Inline path operator -&gt;&gt;</h3><pre>SELECT data FROM courses WHERE data-&gt;&gt;&quot;$.codes&quot; = &#39;[]&#39;;</pre><p>The result as expected, only return records with empty codes</p><pre>+---------------------------------------+<br>| data                                  |<br>+---------------------------------------+<br>| {&quot;name&quot;: &quot;Pharmacology&quot;, &quot;codes&quot;: []} |<br>| {&quot;name&quot;: &quot;Toxicology&quot;, &quot;codes&quot;: []}   |<br>+---------------------------------------+<br>2 rows in set (0.01 sec)</pre><h3>Conclusion</h3><p>JSON type is a really welcome feature that missing at Mysql, also there are some tricks to use correctly the filters and common functions.<br>Also, keep in mind to search for documents with the good performance you may need to create a virtual index.</p><h3>References:</h3><ul><li><a href="https://www.mysqltutorial.org/mysql-json/">https://www.mysqltutorial.org/mysql-json/</a></li><li><a href="https://blog.digital-craftsman.de/select-for-empty-json-array-in-mysql/">https://blog.digital-craftsman.de/select-for-empty-json-array-in-mysql/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53c1194e5efa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Gitlab Pipelines, Build, Tests, and Deploy Private Images (GKE, Pulumi)]]></title>
            <link>https://medium.com/@felipegirotti/gitlab-pipelines-build-tests-and-deploy-private-images-gke-pulumi-480d5d56759b?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/480d5d56759b</guid>
            <category><![CDATA[pipeline]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gitlab-ci]]></category>
            <category><![CDATA[gke]]></category>
            <category><![CDATA[pulumi]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Mon, 30 Aug 2021 12:17:08 GMT</pubDate>
            <atom:updated>2021-08-30T12:17:08.875Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*nceS-_twhusksCXq8-J0xQ.png" /></figure><p>This is the third part of articles to use <a href="https://www.pulumi.com/">Pulumi</a> as infrastructure as Code (IaC) to create and maintain the infrastructure and deploy applications on Kubernetes cluster on the <a href="https://cloud.google.com/">Google Cloud Platform</a> GKE Autopilot and pipelines to automated everything with <a href="https://docs.gitlab.com/ee/ci/pipelines/">GitLab Pipelines</a>. You can check the <a href="https://medium.com/@felipegirotti/gke-autopilot-kubernetes-cluster-with-pulumi-infrastructure-as-code-c74ae8f7ee0f"><strong>first article here!</strong></a></p><p>In this episode, we will create a <a href="https://docs.gitlab.com/ee/ci/pipelines/">GitLab pipeline</a> for a simple Symfony PHP project that will build container images, run lint, unit tests and static analysis code and finally deploy the application in our <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview">GKE Autopilot Cluster</a> with <a href="https://www.pulumi.com/">Pulumi</a>.</p><h3>Configuring the project</h3><p>In our repository, we will change the structure we did in the <a href="https://medium.com/@felipegirotti/gke-autopilot-kubernetes-cluster-with-pulumi-infrastructure-as-code-c74ae8f7ee0f">previews article</a>, to afford the application we will split between infrastructure and application.<br>The structure will be like this: (you can check the final repository files <a href="https://gitlab.com/felipe.girotti/gke-pulumi-pipelines">here</a>)</p><pre>.<br>├── README.md<br>├── application   <br>├── docker<br>│   ├── nginx<br>│   │   ├── Dockerfile<br>│   │   └── application.conf.template<br>│   └── php<br>│       ├── Dockerfile<br>│       └── Dockerfile.base<br>├── infrastructure<br>│   ├── Pulumi.dev.yaml<br>│   ├── Pulumi.yaml<br>│   ├── gcp<br>│   │   └── gke<br>│   │       └── index.ts<br>│   ├── index.ts<br>│   ├── k8s<br>│   │   ├── external-dns<br>│   │   │   ├── index.ts<br>│   │   │   └── namespace.ts<br>│   │   ├── greeting<br>│   │   │   ├── config.ts<br>│   │   │   ├── deployment.ts<br>│   │   │   ├── hpa.ts<br>│   │   │   ├── index.ts<br>│   │   │   ├── ingress.ts<br>│   │   │   ├── namespace.ts<br>│   │   │   ├── pod.ts<br>│   │   │   ├── regsecret.ts<br>│   │   │   └── service.ts<br>│   │   ├── guestbook<br>│   │   │   ├── config.ts<br>│   │   │   ├── index.ts<br>│   │   │   ├── namespace.ts<br>│   │   │   └── redis.ts<br>│   │   └── ingress-nginx<br>│   │       ├── index.ts<br>│   │       └── namespace.ts<br>│   ├── package-lock.json<br>│   ├── package.json<br>│   └── tsconfig.json</pre><p>Basically, we move all files related to infrastructure to the new directory &quot;<strong><em>infrastructure</em></strong>&quot;, create new directories:</p><ul><li><strong><em>application </em></strong>regarding our application (simple Symfony app)</li><li><strong><em>docker</em></strong> regarding the docker files for our application</li><li><strong><em>infrastructure </em></strong>regarding all stuff related to infrastructure as code (Pulumi)</li></ul><h3>Gitlab Container Registry</h3><p>Gitlab offers a free <a href="https://about.gitlab.com/blog/2016/05/23/gitlab-container-registry/"><strong>private container registry</strong></a>, so we have a free git private repository, free pipeline, free container registry in one place, that simplify a lot of work.</p><p>Check this article for the firsts steps: <a href="https://about.gitlab.com/blog/2016/05/23/gitlab-container-registry/"><strong>https://about.gitlab.com/blog/2016/05/23/gitlab-container-registry/</strong></a></p><p>For PHP application is good to have a base image different from your current container image, usually, we need to install extensions and this takes time, the reason we have a file <em>Dockerfile.base, </em>for now, we just build manually and push to the registry.</p><pre>docker build -f docker/php/Dockerfile.base -t registry.gitlab.com/felipe.girotti/gke-pulumi-pipelines/php-base:8.0.8 .</pre><pre>docker push registry.gitlab.com/felipe.girotti/gke-pulumi-pipelines/php-base:8.0.8</pre><h4>Create the registry a secret for Kubernetes</h4><p>Because the container registry is private we need to create a secret in your cluster to make access to pull the image.</p><p>First, let&#39;s configure in the repository the new <a href="https://docs.gitlab.com/ee/user/project/deploy_tokens/index.html"><strong>Deploy Token</strong></a><strong>.<br></strong>Goes to <em>Settings -&gt; Repository -&gt; Deploy Token</em>, create a new user with the permission <strong>read_registry.</strong></p><p>Now we will store those configurations as variables for Pulumi.</p><pre>cd infrastructure<br>pulumi config -s dev set --secret --path registry.password {YOUR_PASSWORD}<br>pulumi config -s dev set --path registry.username k8s<br>pulumi config -s dev set --path registry.url registry.gitlab.com</pre><p>The file:<em> infrastructure/k8s/greeting/config.ts</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0410c7b4803ed41790e33814095056dc/href">https://medium.com/media/0410c7b4803ed41790e33814095056dc/href</a></iframe><p>The secret file: <em>infrastructure/k8s/greeting/regsecret.ts</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e85f2f97bea00a3530558ace3aaed2b7/href">https://medium.com/media/e85f2f97bea00a3530558ace3aaed2b7/href</a></iframe><h3>The Application Pulumi Configuration Kubernetes</h3><p>The new application &quot;<strong><em>greeting</em></strong>&quot; is a simple PHP Symfony application. We have some configurations per environment:</p><ul><li>autoscaling (configuration for our Horizontal Pod Autoscaler)<br>- cpu<br>- memory</li><li>host (the hostname e.g <em>greeting.pulumi-gke.com</em>)</li><li>replicas (configuration for the number of replicas)<br>- min<br>- max</li><li>version (the container image tag)</li></ul><p>The Pulumi.dev.yaml with the configuration of new application &quot;<em>greeting</em>&quot;:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e483cd26aa926d77ba61b24786560242/href">https://medium.com/media/e483cd26aa926d77ba61b24786560242/href</a></iframe><p>Pay attention to lines <strong>10</strong> to <strong>26</strong>, this is the new configuration for <em>greeting</em>.</p><p>Now we have to configure the application for kubernetes, in the folder <strong><em>infrastructure/k8s/guestbook:</em></strong></p><pre>.<br>├── config.ts<br>├── deployment.ts<br>├── hpa.ts<br>├── index.ts<br>├── ingress.ts<br>├── namespace.ts<br>├── pod.ts<br>├── regsecret.ts<br>└── service.ts</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9b92e8a5bf3db60c197ba85870643507/href">https://medium.com/media/9b92e8a5bf3db60c197ba85870643507/href</a></iframe><h3>Gitlab Pipeline</h3><p>Gitlab offers a free tier pipeline that you are able to run since simple to complexity steps without install and maintain one Jenkins.</p><p>Let&#39;s build our first pipeline with the steps:</p><ul><li><strong>build</strong><br>We build the docker images, Nginx, and PHP images and push for our private repository.</li><li><strong>test</strong><br>We doing all necessary tests, linters, static code analysis, unit tests, integration tests, end-to-end tests, security tests, etc…</li><li><strong>release<br></strong>We push the latest container images for our container registry, only after merge in the main (old master)branch</li><li><strong>deploy<br></strong>We push the changes using Pulumi, which could be changed in the application or at the infrastructure level, all changes will be reflected automaticly.</li></ul><h4>Create Service Account GCP</h4><p>We run the infrastructure as code in our pipeline we need to create the Service Account with the permissions to create/edit/delete resources in our cloud provider GCP.</p><p>For this example I put the <strong>owner</strong> role, the best approach is the <a href="https://en.wikipedia.org/wiki/Principle_of_least_privilege">Principle of least privilege</a>, that you only give access to the specific resources, you can check more details of the best practice on GCP <a href="https://cloud.google.com/blog/products/identity-security/dont-get-pwned-practicing-the-principle-of-least-privilege">https://cloud.google.com/blog/products/identity-security/dont-get-pwned-practicing-the-principle-of-least-privilege</a></p><pre>export SA_NAME=&quot;gitlab-ci&quot;<br>export PROJECT_NAME=&quot;YOUR_GCP_PROJECT_NAME&quot;</pre><pre>gcloud iam service-accounts create gitlab-ci \<br>     --description=&quot;GitLab CI Pipelines&quot; \<br>     --display-name=$SA_NAME</pre><pre>gcloud projects add-iam-policy-binding $PROJECT_NAME \<br>    --member=&quot;serviceAccount:<a href="/service/mailto:gitlab-ci@teste-223802.iam.gserviceaccount.com">gitlab-ci@$PROJECT_NAME.iam.gserviceaccount.com</a>&quot; \<br>    --role=&quot;roles/owner&quot;</pre><pre>gcloud iam service-accounts keys create ~/sa-gitlab-private-key.json \<br>    --<a href="/service/mailto:iam-account=gitlab-ci@teste-223802.iam.gserviceaccount.com">iam-account=$SA_NAME@$PROJECT_NAME.iam.gserviceaccount.com</a></pre><pre>cat ~/sa-gitlab-private-key.json | base64</pre><p>Grab the output we will need to create variables</p><h4>Pipeline Variables</h4><p>Now we need to create two variables in our pipeline, one for GCP ServiceAccount another for Pulumi.</p><p>For pulumi access token you can create here: <a href="https://app.pulumi.com/%7BYOUR_USERNAME%7D/settings/tokens">https://app.pulumi.com/{YOUR_USERNAME}/settings/tokens</a></p><p>Now in the <em>Gitlab Repository -&gt; Settings -&gt; CI/CD -&gt; Variables , create two variables:</em></p><ul><li>CI_GOOGLE_APPLICATION_CREDENTIALS (value of base64 encode of our ServiceAccount)</li><li>PULUMI_ACCESS_TOKEN (the pulumi token)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TMYVdyZbsXYVXSgaoRdr1A.png" /><figcaption>GitLab Repository</figcaption></figure><h4>The pipeline configuration file</h4><p>And finally we are able to configure our pipeline, the file <strong><em>.gitlab-ci.yml</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dbe811c37e48e9fac77a929b08f47965/href">https://medium.com/media/dbe811c37e48e9fac77a929b08f47965/href</a></iframe><p>Now we run the pipeline for every Merge Request (Pull Request) for building the images and tests, release images and deploy.</p><p>In at build step we create images with the commit hash, that way we have the guarantee of each pipeline will build unique images, not override the existing ones.</p><p>On test we running the lint, static analysis code and unit tests.</p><p>On the release runs only when merge on the main branch, we just push the lasted tag for container registry repository.</p><p>And finally the deploy run only when merge on main branch, we are using the container image <strong>pulumi/pulumi</strong> that provides the pulumi, gcp and others cli components (you can check the <a href="https://hub.docker.com/r/pulumi/pulumi">details here</a>). We authenticated on the GCP and set the new <strong>version </strong>of the greeting application and run pulumi up.</p><h3>Conclusion</h3><p>Using Pulumi as infrastructure as code combine with the GitLab with container registry and pipelines we automated everything, since create resources on your cloud provider and deploy applications.</p><p>You can check the code in the repository here: <a href="https://gitlab.com/felipe.girotti/gke-pulumi-pipelines">https://gitlab.com/felipe.girotti/gke-pulumi-pipelines</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=480d5d56759b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS EKS autoscaling by default (cluster|hpa)?]]></title>
            <link>https://medium.com/@felipegirotti/aws-eks-autoscaling-by-default-cluster-hpa-aa6e8c08ed69?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa6e8c08ed69</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[ek]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Fri, 20 Aug 2021 22:06:16 GMT</pubDate>
            <atom:updated>2021-08-20T22:06:16.788Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/885/1*mdTM0_Zt6O-vyId1f2cXQw.png" /></figure><p>You are using managed Kubernetes, configure your node group for a min, max, and desired instances, and expected that if need to spin up more nodes (of course respecting the max) will work out-of-the-box.</p><p>In fact, this is expected behavior at least for me because I running a lot of GKE clusters, and GKE dealing with that pretty well. And I surprise for EKS you need to create ASG, Policies, Roles, create service-account and deploy the cluster-autoscaling.</p><p>And if you expected to work with HPA (Horizontal Pod Autscaler) you will be disappointed as well, by default AWS does not provide metrics service, you need to install it manually.</p><p>If you are offering a service managed, that you already have the configs in the node group, why do you need set up other parts? Why AWS does not do it behind the scenes?</p><p>So that is one of the reasons I think the <a href="https://cloud.google.com/">GCP</a> is the best choice to embrace the culture of DevOps because they abstract most part the Ops for you.</p><h3>Metrics Server</h3><p>First, let&#39;s install the <a href="https://github.com/kubernetes-sigs/metrics-server">metrics server</a>, it&#39;s pretty straightforward.</p><pre>kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/<a href="https://github.com/kubernetes-sigs/metrics-server/releases/tag/v0.5.0">v0.5.0</a>/components.yaml</pre><p>And check it&#39;s running:</p><pre>kubectl get apiservice v1beta1.metrics.k8s.io -o json | jq &#39;.status&#39;</pre><h3>Cluster Autoscale</h3><p>If you create the EKS via web console or via cli eksctl they created for you the ASG (Auto Scaling Group) with the launch template, but you need to create policies and role, so let’s do this!</p><p>Identify what the ASG they created for you by CLI:</p><pre>aws autoscaling \     <br>  describe-auto-scaling-groups \     <br>  --query &quot;AutoScalingGroups[? Tags[? (Key==&#39;eks:cluster-name&#39;)]].[AutoScalingGroupName, MinSize, MaxSize,DesiredCapacity]&quot; \     --output table</pre><p>Or you can see in the Auto Scale Group section on web console <a href="https://console.aws.amazon.com/ec2autoscaling/home">https://console.aws.amazon.com/ec2autoscaling/home</a></p><p>Enabling IAM roles in the cluster</p><pre>eksctl utils associate-iam-oidc-provider \<br>     --cluster eksworkshop-eksctl \<br>     --approve</pre><p>Create policy</p><pre>mkdir ~/environment/cluster-autoscaler  </pre><pre>cat &lt;&lt;EoF &gt; ~/environment/cluster-autoscaler/k8s-asg-policy.json <br>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Action&quot;: [<br>        &quot;autoscaling:DescribeAutoScalingGroups&quot;,<br>        &quot;autoscaling:DescribeAutoScalingInstances&quot;,<br>        &quot;autoscaling:DescribeLaunchConfigurations&quot;,<br>        &quot;autoscaling:DescribeTags&quot;,<br>        &quot;autoscaling:SetDesiredCapacity&quot;,<br>        &quot;autoscaling:TerminateInstanceInAutoScalingGroup&quot;,<br>        &quot;ec2:DescribeLaunchTemplateVersions&quot;<br>      ],<br>      &quot;Resource&quot;: &quot;*&quot;,<br>      &quot;Effect&quot;: &quot;Allow&quot;<br>    }<br>  ]<br>} <br>EoF</pre><pre>aws iam create-policy   \<br>   --policy-name k8s-asg-policy \<br>   --policy-document file://~/environment/cluster-autoscaler/k8s-asg-policy.json</pre><p>Create a role and kubernetes ServiceAccount</p><pre>eksctl create iamserviceaccount \<br>     --name cluster-autoscaler \<br>     --namespace kube-system \<br>     --cluster ${CLUSTER_NAME} \<br>     --attach-policy-arn &quot;arn:aws:iam::${ACCOUNT_ID}:policy/k8s-asg-policy&quot; \<br>     --approve \<br>     --override-existing-serviceaccounts</pre><p>Check the ServiceAccount</p><pre>kubectl -n kube-system describe sa cluster-autoscaler</pre><p>You can see the annotations you have the role</p><p>Now we need to deploy <a href="https://github.com/kubernetes/autoscaler/tree/master/charts/cluster-autoscaler">cluster-autoscale</a>, for that we will use helm chart.</p><pre>helm repo add autoscaler <a href="https://kubernetes.github.io/autoscaler">https://kubernetes.github.io/autoscaler</a></pre><pre>helm install cluster-autoscaler autoscaler/cluster-autoscaler \<br>--namespace kube-system \<br>--set &#39;autoDiscovery.clusterName&#39;=&lt;CLUSTER NAME&gt; \<br>--set &#39;awsRegion&#39;=&lt;AWS_REGION&gt;</pre><p>But the trick for making it work is, your ASG and your EC2 instances MUST have 2 special TAGs</p><ul><li><strong>k8s.io/cluster-autoscaler/enabled </strong>= true</li><li><strong>k8s.io/cluster-autoscaler/&lt;YOUR_CLUSTER_NAME&gt; </strong>=<strong> </strong>owned</li></ul><p>First, goes to web console, ASG and edit the tags adding those.</p><p>Secondly, in the web console page for ASG the tag &quot;Instance Management&quot; you have a list of the EC2 instances, goes for each one edit the tags adding those listed above.</p><p>Finally, you can see the logs without the &quot;<strong>no node group config</strong>&quot;</p><h3>Credits</h3><ul><li><a href="https://www.eksworkshop.com/beginner/080_scaling/">https://www.eksworkshop.com/beginner/080_scaling/</a></li><li><a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/containers-provision-eks-clusters-terraform/helm-add-ons.html">https://docs.aws.amazon.com/prescriptive-guidance/latest/containers-provision-eks-clusters-terraform/helm-add-ons.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa6e8c08ed69" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Install Ingress-Nginx and ExternalDNS with Pulumi on GKE Autopilot]]></title>
            <link>https://medium.com/@felipegirotti/install-ingress-nginx-and-externaldns-with-pulumi-on-gke-autopilot-6417c13f99ce?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/6417c13f99ce</guid>
            <category><![CDATA[ingress]]></category>
            <category><![CDATA[pulumi]]></category>
            <category><![CDATA[gke]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <category><![CDATA[external-dns]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Sat, 31 Jul 2021 18:04:04 GMT</pubDate>
            <atom:updated>2021-08-02T13:29:57.409Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-KcRHMoQUylpblYSHNvXMw.jpeg" /></figure><p>This is the second part of articles to use <a href="https://www.pulumi.com/">Pulumi</a> as infrastructure as Code (IaC) to create and maintain the infrastructure and deploy applications on Kubernetes cluster on the <a href="https://cloud.google.com/">Google Cloud Platform</a> GKE Autopilot and pipelines to automated everything with <a href="https://docs.gitlab.com/ee/ci/pipelines/">GitLab Pipelines</a>. You can check the <a href="https://medium.com/@felipegirotti/gke-autopilot-kubernetes-cluster-with-pulumi-infrastructure-as-code-c74ae8f7ee0f"><strong>first article here!</strong></a></p><p>We will see how to install <a href="https://kubernetes.github.io/ingress-nginx/"><strong>ingress-nginx</strong></a><strong>,</strong> <a href="https://github.com/kubernetes-sigs/external-dns"><strong>external-dns</strong></a>, <a href="http://cloudflare.com/"><strong>CloudFlare</strong></a> to configure domains and create a Ingress for our application guestbo</p><h3>Ingress</h3><p><a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">Kubernetes Ingress</a> exposes HTTP/HTTPS, SSL/TLS from outside the cluster to services. We can use traffic routes, load balancers, access control, websockets, etc. All those things with one integration with service load balancer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*KIqn4POmuY5-McltuvarEQ.png" /><figcaption><a href="https://kubernetes.io/docs/concepts/services-networking/ingress/">https://kubernetes.io/docs/concepts/services-networking/ingress/</a></figcaption></figure><h3>ExternalDNS</h3><p><a href="https://github.com/kubernetes-sigs/external-dns">ExternalDNS</a> allows you to control DNS records dynamically via Kubernetes resources in a DNS provider-agnostic way.</p><p>This means when you install a new service in the cluster, this will configure automatically the DNS for the new service, adding the new entry <strong>A</strong> in the DNS for this service.</p><p>We will use <a href="https://www.cloudflare.com/">Cloudflare</a> to control our DNS, Cloudflare provides much more than manager DNS, they provide free SSL certificates, ant-DDOS, CDN, and so on, all those features in the free tier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*nJz7cvVvLxoqv3gRfBEWOQ.png" /></figure><h3>Configuration for Ingress-Nginx</h3><p>So first we will configure<a href="https://kubernetes.github.io/ingress-nginx/"> Ingress-Nginx</a> with helm chart and Pulumi.</p><p>Our project structure will be like this (please check the <a href="https://medium.com/@felipegirotti/gke-autopilot-kubernetes-cluster-with-pulumi-infrastructure-as-code-c74ae8f7ee0f"><strong>first article</strong></a> to have context):</p><pre>├── Pulumi.dev.yaml<br>├── Pulumi.yaml<br>├── gcp<br>│   └── gke<br>│       └── index.ts<br>├── index.ts<br>├── k8s<br>│   ├── external-dns<br>│   │   ├── index.ts<br>│   │   └── namespace.ts<br>│   ├── guestbook<br>│   │   ├── config.ts<br>│   │   ├── index.ts<br>│   │   ├── namespace.ts<br>│   │   └── redis.ts<br>│   └── ingress-nginx<br>│       ├── intex.ts<br>│       └── namespace.ts<br>├── package-lock.json<br>├── package.json<br>└── tsconfig.json</pre><p>The only variable we will use for helm chart is the version of helm chart, for that we just need add this information on the Pulumi.dev.yaml or using the cli for that:</p><pre>pulumi config -s dev set ingressNginxChartVersion 3.34.0</pre><p>The files of folder <strong>k8s/ingress-nginx</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/45f978442d048b9f937b74e5de0ac691/href">https://medium.com/media/45f978442d048b9f937b74e5de0ac691/href</a></iframe><h3>Configuration for ExternalDNS</h3><h4>Configure CloudFlare</h4><p>First, you need to create an account in <a href="https://www.cloudflare.com/">CloudFlare</a> for your domain and create the token on Cloudflare, it&#39;s pretty straightforward.<br>Goes to <a href="https://dash.cloudflare.com/profile/api-tokens"><strong>My Profile -&gt; Api Tokens</strong></a> click to create token button select to Edit zone DNS template and that it!</p><h4>Handle with secrets on Pulumi</h4><p>Now we use another interesting feature on Pulumi, the Secrets, how we can handle that. In the docs of Pulumi you have the entire information: <a href="https://www.pulumi.com/docs/intro/concepts/secrets/">https://www.pulumi.com/docs/intro/concepts/secrets/</a></p><p>For our project we just create a new secret:</p><pre>pulumi config -s dev set --secret externalDnsCloudFlareToken {YOUR_TOKEN_HERE}</pre><p>P.S (In the future article we will see how to use <a href="https://cloud.google.com/secret-manager">Secrets Manager on GCP</a> with Pulumi).</p><p>And let&#39;s edit our Pulumi.dev.yaml file to adding more variables for external-dns service (domainFilters and version), the final version of yaml is:</p><pre>config:<br>  gcp:project: {CHANGE_FOR_YOUR_GCP_PROJECT_NAME}<br>  infrastructure:externalDns:<br>    domainFilters:<br>      - pulumi-gke.com // change for your domain<br>    version: 5.1.4<br>  infrastructure:externalDnsCloudFlareToken:<br>    secure: {SECRET_BY_PULUMI}<br>  infrastructure:gke-min-version: &quot;1.19&quot;<br>  infrastructure:guessbook:<br>    replicas: 1<br>  infrastructure:ingressNginxChartVersion: 3.34.0</pre><p>The files of folder <strong>k8s/external-dns (</strong>to more details of helm chart values, you can check here:<strong> </strong><a href="https://github.com/bitnami/charts/tree/master/bitnami/external-dns">https://github.com/bitnami/charts/tree/master/bitnami/external-dns</a>)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1162e9e59f317f67620e25a209e7fd33/href">https://medium.com/media/1162e9e59f317f67620e25a209e7fd33/href</a></iframe><p>The last step is to change your file <strong>/index.ts </strong>to get references of those 2 new applications.</p><pre>export * from &#39;./gcp/gke&#39;;<br>export * from &#39;./k8s/guessbook&#39;;<br>require(&#39;./k8s/ingress-nginx&#39;);<br>require(&#39;./k8s/external-dns&#39;);</pre><p>And run the up!</p><pre>pulumi up -s dev</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yYiuV2nO4C6ErJR0gqsU5A.gif" /></figure><h3>Configure the Ingress for Guestbook application</h3><p>Now we just need to configure the ingress to expose your guestbook application by the domain.</p><p>We will create a new entry in your Pulumi.dev.yaml file for the domain, that way each stack that represent each environment could have the different domains.</p><pre>pulumi config -s dev set --path  guestbook.host guestbook.pulumi-gke.com</pre><p>Change the <strong>k8s/guestbook/config.ts</strong> to identify this new config value <strong>host</strong></p><pre>import * as pulumi from &#39;@pulumi/pulumi&#39;;<br><br>interface GuestBookConfiguration {<br>    replicas: number;<br>    <strong>host</strong>: <strong>string</strong>;<br>}<br><br>const <em>config</em><strong><em> </em></strong>= new pulumi.Config();<br>const <em>guestbookConfig</em>: GuestBookConfiguration = <em>config</em>.requireObject(&#39;guestbook&#39;);<br><br>export { <em>config</em>, <em>guestbookConfig</em><strong><em> </em></strong>};</pre><p>Modify the <strong>service </strong>to remove the type, we don&#39;t need anymore the load-balance service, this will handle by ingress-nginx that contain the service load-balance.<br>When we try just to remove type or change the type of service for ClusterIP we got a error from Pulumi.</p><pre>the Kubernetes API server reported that &quot;guestbook/guestbook-p3bune9t&quot; failed to fully initialize or become live: Service &quot;guestbook-p3bune9t&quot; is invalid: spec.ports[0].nodePort: Forbidden: may not be used when `type` is &#39;ClusterIP&#39;</pre><p>To solve it, firs we just comment the service on the file <strong>k8s/guestbook/index.ts </strong>and run <strong>pulumi up -s dev</strong></p><pre>// const service = deployment.createService({<br>//     ports: [<br>//         {<br>//             port: 80,<br>//         },<br>//     ],<br>//     type: k8sx.types.ServiceType.LoadBalancer,<br>// });<br><br>// export const response = {<br>//     ip: service.endpoint<br>// };</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/943/1*z-b8pAuwvXsNyZ-mvqXiFw.png" /></figure><p>Now we are able to recreate service without issues.</p><p>And create the new Ingress for the guestbook application, the final version of file <strong>k8s/guestbook/index.ts</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/23a48f2f316c30905be65057eafb5941/href">https://medium.com/media/23a48f2f316c30905be65057eafb5941/href</a></iframe><p>And again the command <strong>pulumi up -s dev</strong></p><p>And now finally we can check the logs on the external-dns</p><pre>kubectl logs -n external-dns deployment/external-dns</pre><p>You see something like that:</p><pre>time=&quot;2021-07-31T17:45:53Z&quot; level=info msg=&quot;Changing record.&quot; action=CREATE record=guestbook.pulumi-gke.com ttl=1 type=A zone=e2018573b2f2b81180a455b0a969784f<br>time=&quot;2021-07-31T17:45:53Z&quot; level=info msg=&quot;Changing record.&quot; action=CREATE record=guestbook.pulumi-gke.com ttl=1 type=TXT zone=e2018573b2f2b81180a455b0a969784f</pre><p>And you can check on the web console on CloudFlare the entries were created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NENWwWFJkfAJblywT7zYsg.png" /></figure><p>And access the domain to check guestbook application is working, the best thing we do not need configure SSL, CloudFlare does it for us!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ciQDSMfPdo-3Z1upemQgjQ.png" /></figure><h3>Conclusion</h3><p>Using Pulumi as infrastructure as code we can control better what the dependencies we need in our cluster, ingress-nginx and external-dns are a perfect combination to automated the process to adding entries on DNS.</p><p>You can check all files in our repository: <a href="https://gitlab.com/felipe.girotti/gke-pulumi-pipelines/-/tree/v0.2.0">https://gitlab.com/felipe.girotti/gke-pulumi-pipelines/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6417c13f99ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GKE Autopilot Kubernetes Cluster with Pulumi Infrastructure as Code]]></title>
            <link>https://medium.com/@felipegirotti/gke-autopilot-kubernetes-cluster-with-pulumi-infrastructure-as-code-c74ae8f7ee0f?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/c74ae8f7ee0f</guid>
            <category><![CDATA[autopilot]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[gke]]></category>
            <category><![CDATA[pulumi]]></category>
            <category><![CDATA[infrastructure-as-code]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Sat, 17 Jul 2021 20:23:17 GMT</pubDate>
            <atom:updated>2021-07-31T16:49:28.682Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-KcRHMoQUylpblYSHNvXMw.jpeg" /></figure><p>This is the first part of articles to use <a href="https://www.pulumi.com/">Pulumi</a> as infrastructure as Code (IaC) to create and maintain the infrastructure and deploy applications on Kubernetes cluster on the <a href="https://cloud.google.com/">Google Cloud Platform</a> GKE Autopilot and pipelines to automated everything with <a href="https://docs.gitlab.com/ee/ci/pipelines/">GitLab Pipelines</a>.</p><p><a href="https://www.pulumi.com/">Pulumi</a> is one of best tool to use as infrastructure as code. This has been empowering and help many DevOps and Engineers teams around the world turn our lives better, less time on provisioning and operations toil, and provide to platform team delivery self-service infra products in efficient matters.</p><h3>Installing required tools</h3><ul><li>Google Cloud SDK <a href="https://cloud.google.com/sdk/">https://cloud.google.com/sdk</a></li><li>Pulumi <a href="https://www.pulumi.com/docs/get-started/install/">https://www.pulumi.com/docs/get-started/install/</a></li><li>Kubectl <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">https://kubernetes.io/docs/tasks/tools/install-kubectl/</a></li><li>Node <a href="https://nodejs.org/en/download/">https://nodejs.org/en/download/</a> (we will use Typescript as language on Pulumi)</li></ul><h3>Start with Pulumi</h3><p>Let&#39;s start with a gcp template</p><pre>pulumi new gcp-typescript</pre><p>Follow the steps in the cli for name of project, description, the stack and GCP project name</p><p>Pulumi will install all dependencies we need to handle GCP resources, but still we need Kubernetes modules, let&#39;s install.</p><pre>npm install --save <a href="http://twitter.com/pulumi/kubernetes">@pulumi/kubernetes</a> <a href="http://twitter.com/pulumi/kubernetesx">@pulumi/kubernetesx</a></pre><p>The <em>index.ts</em> file is the example to create a bucket, let&#39;s play with that to familiarize with Pulumi.<br>Run the command in our terminal:</p><pre>pulumi up</pre><figure><img alt="pulumi up details" src="https://cdn-images-1.medium.com/max/960/1*nFyFFbveCS2T0NSDR6nrvw.gif" /></figure><p>Just run remove the test</p><pre>pulumi destroy</pre><h3>Organize directory structure</h3><p>Now we will to organize the directories to separate GCP resources and kubernetes resources.</p><p>Looks like this:</p><pre>├── Pulumi.dev.yaml<br>├── Pulumi.yaml<br>├── gcp<br>│   └── gke<br>│       └── index.ts<br>├── index.ts<br>├── k8s<br>│   └── guestbook<br>│       ├── config.ts<br>│       ├── index.ts<br>│       ├── namespace.ts<br>│       └── redis.ts<br>├── package-lock.json<br>├── package.json<br>└── tsconfig.json</pre><h3>Configure the GKE Autopilot Cluster</h3><p>The <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview"><strong>Autopilot</strong></a> we have the benefits of Google will handle our nodes, it&#39;s a fully managed Kubernetes cluster. <br>Of course there some trade-offs, for example you are not allowed to use Istio as service mesh, cert-manager and others that need <strong>mutatingwebhookconfigurations</strong>, you can check all features and limitations here: <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview">https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview</a></p><p>First, we will setup variables on the file Pulumi.dev.yaml, the variable for the min gke kubernetes version in your case <strong>1.19 </strong>(this is the stable version on the date of the article created).<br>Pay attention to the name <strong>infrastructure: </strong>this is must be the same name as your Pulumi project name in the file<strong> Pulumi.yaml</strong></p><pre>config:<br>  gcp:project: {REPLACE_FOR_YOUR_GCP_PROJECT}<br>  infrastructure:gke-min-version: &#39;1.19&#39;<br>  </pre><p>So let&#39;s edit your<strong><em> gcp/gke/index.ts</em></strong> file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/99a4438ddaeb543a03a4f7f7b85e08d6/href">https://medium.com/media/99a4438ddaeb543a03a4f7f7b85e08d6/href</a></iframe><p>And the root<strong><em> index.ts</em></strong></p><pre>export * from &#39;./gcp/gke&#39;;</pre><p>and run the command up</p><pre>pulumi up</pre><p>This process could take a few minutes to create a GKE cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TY9uIazssiGHdASWhkrtfg.gif" /></figure><h3>Configure the application</h3><p>For the exemplification, we will use the <a href="https://cloud.google.com/kubernetes-engine/docs/tutorials/guestbook"><strong>guestbook</strong></a> application (in the next episodes we will adding more complex applications).</p><h4>Configure variables:</h4><p>As the same variable for <strong>gke-min-version </strong>for the cluster, we will create just one, the number of replicas for the application.</p><pre>pulumi config -s dev set --path  guestbook.replicas 1</pre><p>To simplify the process of write manifests files we will use the package <a href="https://github.com/pulumi/pulumi-kubernetesx"><strong>pulumi/kubernetesx</strong></a><strong>.<br></strong>Remember the structure of directories, the files are on the <strong>k8s/guestbook</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bda2d64fcb31f87b8a1014c1c3255262/href">https://medium.com/media/bda2d64fcb31f87b8a1014c1c3255262/href</a></iframe><p>Change the file <strong>index.ts</strong> to add the entry for the application</p><pre>export * from &#39;./gcp/gke&#39;;<br>export * from &#39;./k8s/guestbook&#39;;</pre><p>And run up command:</p><pre>pulumi up</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jRmvhbz93R7G6VcIUwVlgQ.gif" /></figure><p>The output response contains the IP to access the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*YMvwFfloiMcgtKpcuddCbw.png" /></figure><p>To clean up just run the destroy command:</p><pre>pulumi destroy</pre><h3>Conclusion</h3><p>This is the first introduction of what we can able to do with Pulumi as infrastructure as code.<br>In the next episodes, we will explore more functionalities of Pulumi combine the pipeline on GitLab.</p><p>Here you can see a free and good Pulumi Crash Course on Youtube: <a href="https://www.youtube.com/channel/UCSt5XpbeUuRQ-CndGtYFL8w">https://www.youtube.com/channel/UCSt5XpbeUuRQ-CndGtYFL8w</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c74ae8f7ee0f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Elasticsearch filter field array more than zero]]></title>
            <link>https://medium.com/@felipegirotti/elasticsearch-filter-field-array-more-than-zero-8d52d067d3a0?source=rss-9b1de7721140------2</link>
            <guid isPermaLink="false">https://medium.com/p/8d52d067d3a0</guid>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Felipe Girotti]]></dc:creator>
            <pubDate>Thu, 01 Aug 2019 01:20:11 GMT</pubDate>
            <atom:updated>2019-08-01T15:03:16.919Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*Co95dG0NmGfL-vGMSBtLWQ.png" /></figure><p><a href="https://www.elastic.co/products/enterprise-search">Elasticsearch</a> is one of the best tools for searching, in my job we are using intensive it (since version <strong>6.*</strong> to <strong>7.*</strong>).</p><p>Usually, we need to filter the array, just to return when have values to array.<br>If the array it consists of a list of strings and the mapping of the string it&#39;s the default, like this bellow:</p><pre>{<br>  &quot;semantics&quot;: {<br>    &quot;type&quot;: &quot;text&quot;,<br>    &quot;fields&quot;: {<br>      &quot;keyword&quot;: {<br>        &quot;type&quot;: &quot;keyword&quot;,<br>        &quot;ignore_above&quot;: 256<br>      }<br>    }<br>  }<br>}</pre><p>When you attempt to filter by script like this:</p><pre>{<br>  &quot;query&quot;: {<br>    &quot;bool&quot;: {<br>      &quot;filter&quot;: {<br>        &quot;script&quot;: {<br>          &quot;script&quot;: {<br>            &quot;source&quot;: &quot;doc[&#39;<strong>semantics</strong>&#39;].values.length &gt; 0&quot;,<br>            &quot;lang&quot;: &quot;painless&quot;<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>You got an error:</p><pre>org.elasticsearch.index.mapper.TextFieldMapper$TextFieldType.fielddataBuilder(TextFieldMapper.java:779)</pre><pre>&quot;doc[&#39;<strong>semantics</strong>&#39;].values.length &gt; 0&quot;,<br>&quot;    ^---- HERE&quot;</pre><p>The error consists because the values it&#39;s a text. The fix&#39;s super simple, just use the <strong>keyword</strong>, in the name of the field, so the complete filter:<br><strong>IMPORTANT! this is for version 6.*</strong></p><pre>{<br>  &quot;query&quot;: {<br>    &quot;bool&quot;: {<br>      &quot;filter&quot;: {<br>        &quot;script&quot;: {<br>          &quot;script&quot;: {<br>            &quot;source&quot;: &quot;doc[&#39;<strong>semantics.keyword</strong>&#39;].values.length &gt; 0&quot;,<br>            &quot;lang&quot;: &quot;painless&quot;<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p><strong>For version 7.*</strong></p><pre>{<br>  &quot;query&quot;: {<br>    &quot;bool&quot;: {<br>      &quot;filter&quot;: {<br>        &quot;script&quot;: {<br>          &quot;script&quot;: {<br>            &quot;source&quot;: &quot;doc[&#39;<strong>semantics.keyword</strong>&#39;].length &gt; 0&quot;,<br>            &quot;lang&quot;: &quot;painless&quot;<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>I hope this &quot;trick&quot; help you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d52d067d3a0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>