<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by David Garcia on Medium]]></title>
        <description><![CDATA[Stories by David Garcia on Medium]]></description>
        <link>https://medium.com/@own3dh2so4?source=rss-1a957b128e48------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*URuzRYQbxbQnjIf7</url>
            <title>Stories by David Garcia on Medium</title>
            <link>https://medium.com/@own3dh2so4?source=rss-1a957b128e48------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:51:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@own3dh2so4/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Isolation Levels in Databases: From Dirty Reads to Serializable Transactions]]></title>
            <link>https://medium.com/@own3dh2so4/understanding-isolation-levels-in-databases-from-dirty-reads-to-serializable-transactions-e40e590a95d4?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/e40e590a95d4</guid>
            <category><![CDATA[database-isolation]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[acid]]></category>
            <category><![CDATA[dbms]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Fri, 11 Jul 2025 14:44:48 GMT</pubDate>
            <atom:updated>2025-07-11T14:50:19.905Z</atom:updated>
            <content:encoded><![CDATA[<p>The main objective of this post is to review transaction isolation levels. My goal is to provide a clear and simple guide with the main terms clearly explained.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GN9MlAtN_xFE_9bgmo35pg.png" /></figure><h4>🔁 What is a transaction</h4><p>A <strong>transaction</strong> is a logical unit of work that groups one or more operations. It should follow <strong>ACID </strong>properties:</p><ul><li><em>Atomicity</em>: All or nothing.</li><li><em>Consistency</em>: The database rules must always be preserved.</li><li><em>Isolation</em>: Concurrent transactions should not interfere with each other.</li><li><em>Durability</em>: Once committed, the changes persist even if the system crashes.</li></ul><h4>💣 Common concurrency issues</h4><p><strong>Dirty reads<br></strong>A transaction reads data that has been <strong>modified but not committed</strong> by another transaction. Example:</p><ul><li>T1 updates the balance from $100 to $50 (no commit yet).</li><li>T2 reads the balance: sees €50.</li><li>T1 rolls back → the correct balance is still €100, but T2 read an invalid value.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/631/1*6PmLQWqEGxkoJ4iKXfKSiQ.png" /><figcaption>example of dirty read</figcaption></figure><p><strong>Non-repeteable reads<br></strong>A transaction <strong>reads the same row twice and gets different values</strong>, because another transaction modified it between reads. Example:</p><ul><li>T1 reads balance: $100.</li><li>T2 updates balance to $50 and commits.</li><li>T1 reads again → sees $50. Data changed during T1’s transaction.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*J_bHYDf3_0-PHxwrRyZU0w.png" /><figcaption>example of non-repeteable read</figcaption></figure><p><strong>Phantom reads<br></strong>A transaction <strong>executes the same query twice and sees a different number of rows</strong> due to another transaction <strong>inserting or deleting matching records</strong>. Example:</p><ul><li>T1: SELECT * from bank_card_expenses WHERE user_id = 1 → 3 rows.</li><li>T2 inserts a new bank_card_expenses with user_id = 1 and commits.</li><li>T1 re-executes the same query → now 4 rows. The new row is a phantom.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*GQ8X7k5w1hCTzEPgVY_2AQ.png" /><figcaption>example of phantom read</figcaption></figure><p><strong>Serialization Anomaly<br></strong>A situation where the outcome of concurrent execution<strong> could not be produced by any serial (one-by-one) execution</strong> of the same transactions. Example:<br>Imagine a rule where a user’s bank balance must never go negative.</p><ul><li>T1 checks the user’s balance: it sees $100</li><li>T2 also checks the balance and sees $100</li><li>T1 deducts $60 from the account</li><li>T2 deducts $80 from the account</li><li>Result: the balance becomes -$40, violating the system’s constraint.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*HMinXl0r5RgtNuHeCaArig.png" /><figcaption>example serialization anomaly</figcaption></figure><h4>⚮ Isolation levels</h4><p>Isolation levels define <strong>how strictly transactions are separated</strong> from each other.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/75c5678bd934f257f594ef5aaee845b7/href">https://medium.com/media/75c5678bd934f257f594ef5aaee845b7/href</a></iframe><p><strong>Read Uncommitted<br></strong>Allows reading <strong>uncommitted</strong> data. Fastest, but least safe. Prone to: <strong>Dirty Reads</strong>, example:</p><ul><li>T1: UPDATE accounts SET balance = balance - 100 WHERE id = 1 (not committed).</li><li>T2: SELECT balance FROM accounts WHERE id = 1 → sees balance reduced.</li><li>T1 rolls back → T2 saw a value that never truly existed.</li></ul><p><strong>Read Committed</strong><br>Only reads <strong>committed</strong> data. But data can still change between reads in the same transaction. Prone to: <strong>Non-Repeatable Reads</strong>, example:</p><ul><li>T1: reads balance → €100.</li><li>T2: updates balance to €50 and commits.</li><li>T1 reads again → sees €50.</li></ul><p><strong>Repeatable Read<br></strong>Prevents changes to <strong>rows read</strong> by a transaction. But <strong>new matching rows</strong> may appear → phantoms. Prone to: <strong>Phantom Reads</strong>, example:</p><ul><li>T1: SELECT * from bank_card_expenses WHERE user_id = 1 → returns 2 rows.</li><li>T2 inserts a new bank_card_expensesfor the same customer and commits.</li><li>T1 runs the same query again → now sees 3 rows.</li></ul><p><strong>Serializable<br></strong>Strictest level: transactions behave as if executed <strong>one at a time</strong>. Prevents <strong>dirty</strong>, <strong>non-repeatable</strong>, and <strong>phantom</strong> reads. Guarantees full isolation at the cost of performance, example:</p><ul><li>T1: SELECT SUM(stock) FROM products WHERE category = &#39;books&#39;.</li><li>T2: tries to INSERT INTO products ... for that category.</li><li>T2 is blocked until T1 finishes → no phantom rows.</li></ul><h4>🏁Conclusion</h4><p>As we’ve seen, each isolation level has advantages and disadvantages, and the application’s use case depends largely on the application. We’ll likely want the highest possible level of isolation for handling financial transactions, but for tracking the number of likes on a photo, the lowest level is useful.</p><p>Before concluding, I’d like to recommend some sources I used to base this article:</p><ul><li><a href="https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/">Designing Data-Intensive Applications</a> (this book is a must!)</li><li><a href="https://redis.io/glossary/acid-transactions/">ACID Transaction — Redis blog</a></li><li><a href="https://www.cockroachlabs.com/blog/sql-isolation-levels-explained/">No Dirty Reads: Everything you always wanted to know about SQL isolation levels (but were too afraid to ask)</a> — Cockroach Labs blog</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e40e590a95d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FOSDEM 2025 My Review]]></title>
            <link>https://medium.com/joor-engineering/fosdem-2025-my-review-b85119e08416?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/b85119e08416</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[fosdem]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Mon, 17 Feb 2025 09:37:37 GMT</pubDate>
            <atom:updated>2025-02-17T09:37:37.853Z</atom:updated>
            <content:encoded><![CDATA[<p>My name is David Garcia, Staff Engineer at JOOR, and a few weeks ago, I had the privilege of attending FOSDEM once again — this time to celebrate the conference’s 25th anniversary! <a href="https://medium.com/joor-engineering/fosdem-2024-my-review-d7a4c7f3c547">As I did last year</a>, I’ll share my key takeaways from the event…</p><h4>What is FOSDEM?</h4><p><strong>FOSDEM</strong> (Free and Open Source Software Developers’ European Meeting) is an annual conference held in Brussels, Belgium, dedicated to <strong>free and open-source software</strong>. The event features various talks covering various topics, including programming languages, security, cloud computing, and embedded systems. FOSDEM is free to attend and is known for its community-driven, <strong>non-commercial atmosphere</strong>.</p><p>From my perspective, it’s a must-attend event every year to <strong>stay informed</strong> about what’s happening in the <strong>open-source world</strong>. It not only helps you stay up to date but also allows you to see how <strong>other engineering</strong> teams <strong>have tackled various challenges</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sQW4TuEYbDGNyVMS2Uywww.jpeg" /><figcaption>FOSDEM opening</figcaption></figure><h4>How to prepare for it?</h4><p>To prepare for FOSDEM, the first thing to consider is <strong>your profile and interests</strong>. In my case, I’m particularly passionate about <strong>backend</strong> technologies and <strong>infrastructure</strong>, so I tend to prioritize those tracks.</p><p>Additionally, as I mentioned last year, to organize myself and decide which tracks to attend and what talks are available in each, I use an excellent Android app called <a href="https://play.google.com/store/apps/details?id=be.digitalia.fosdem&amp;hl=es_419"><strong>FOSDEM Companion</strong></a>.</p><p>Once we’ve decided which tracks we want to attend, we need to check the rooms where they take place. It’s also important to consider that not all rooms are the same size, and the number of attendees for each talk can vary. If there’s a talk you don’t want to miss, I recommend attending the previous session on the same track to secure your spot.</p><h4>My 2025 favourite talks</h4><p>This year, as in previous ones, the tracks I prioritized were <strong>Containers, Monitoring, Postgres, Python</strong>, and a new track they introduced: <strong>Cloud Native Databases</strong>. Based on this my favorite talks:</p><ul><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-5036-reusing-postgresql-codebase-in-a-distributed-sql-architecture-yugabytedb-/">Reusing PostgreSQL codebase in a Distributed SQL Architecture (YugabyteDB)</a>: I found this talk really interesting because it explained how they reused PostgreSQL’s code for their new database engine.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-4759-migrating-massive-aurora-and-mysql-databases-to-vitess-kubernetes-clusters-with-near-zero-downtime/">Migrating Massive Aurora and MySQL Databases to Vitess Kubernetes Clusters with Near-Zero Downtime</a>: They provide a very visual and instructive explanation of how to perform a database migration. Although they focus on their own service, the general principles are applicable to any migration.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-5956-tracing-the-internals-of-a-cloud-native-database/">Tracing the Internals of a Cloud-Native Database</a>In this case, they talk about how the tracing system works within ClickHouse.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-4958-distributed-sql-technologies-raft-lsm-trees-time-and-more/">Distributed SQL Technologies: Raft, LSM Trees, Time, and More</a>: What I found most interesting about this talk was its format. They discussed how they tackled various database challenges from the perspective of <strong>TiDB</strong> and <strong>YugabyteDB</strong>.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-5743-distributed-databases-essential-or-optional-/">Distributed Databases: Essential or Optional?</a>: A short talk, but since it’s given by Peter Zaitsev, it’s a must-attend.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-4146-discovering-the-magic-behind-opentelemetry-instrumentation/">Discovering the Magic Behind OpenTelemetry Instrumentation</a>: Yet another talk about OpenTelemetry, but this time, it was well-explained and very visual.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-5502-the-performance-impact-of-auto-instrumentation/">The performance impact of auto-instrumentation</a>: The analysis was very interesting, although the conclusions were a bit strange, and the way the data was presented felt somewhat disorganized.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-5028-zero-code-distributed-traces-for-any-programming-language/">Zero-Code Distributed Traces for any programming language</a>: My favorite this year within the monitoring track was a talk about how to add traces to any application using eBPF by embedding certain metadata within the information traveling through TCP/IP.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-4603-anatomy-of-table-level-locks-in-postgresql/">Anatomy of Table-Level Locks in PostgreSQL</a>: A super interesting talk about the locks PostgreSQL applies when making schema changes and how we could perform these changes with fewer locks.</li><li><a href="https://fosdem.org/2025/schedule/event/fosdem-2025-4852-how-browsers-really-load-web-pages/">How browsers REALLY load Web pages</a>: It demystifies a lot of what happens every time a browser loads a webpage.</li></ul><p>And these are the talks I would highlight from this year, 2025. I still have a few saved that I haven’t watched yet, most of which haven’t had their videos uploaded, but I check them every day. Honestly, I’m already looking forward to attending again next year!</p><h4><strong>Next steps</strong></h4><p>Attending FOSDEM is a great personal experience, but it doesn’t end there, the real impact comes from what we do next. To maximize the benefits of this experience, we should share key takeaways with our teammates, and encourage discussions about open-source adoptions. Evangelizing open-source principles within our teams is essential to fostering a culture of collaboration, transparency, and innovation.</p><p>We also have to keep trying to attend these types of events. The next one on my radar is the <a href="https://www.opensouthcode.org/conferences/opensouthcode2025">OpenSouthCode</a> held in Malaga. I hope to find you there :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b85119e08416" width="1" height="1" alt=""><hr><p><a href="https://medium.com/joor-engineering/fosdem-2025-my-review-b85119e08416">FOSDEM 2025 My Review</a> was originally published in <a href="https://medium.com/joor-engineering">JOOR Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How we integrate Zendesk SSO login into JOOR]]></title>
            <link>https://medium.com/joor-engineering/how-we-integrate-zendesk-sso-login-into-joor-4394aeef89d2?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/4394aeef89d2</guid>
            <category><![CDATA[zendesk-guide]]></category>
            <category><![CDATA[sso]]></category>
            <category><![CDATA[zendesk]]></category>
            <category><![CDATA[zendesk-integration]]></category>
            <category><![CDATA[development]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Wed, 14 Aug 2024 07:54:57 GMT</pubDate>
            <atom:updated>2024-08-14T07:54:57.089Z</atom:updated>
            <content:encoded><![CDATA[<p>Like other companies, JOOR integrates Zendesk to improve customer support efficiency. It provides a unified platform for managing customer inquiries across various channels (email, chat, phone, social media) and helps streamline communication, automate repetitive tasks, and track customer interactions. This enhances the customer experience and allows companies to resolve issues more quickly and effectively.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*or9hWBLW9nXR_X22.png" /></figure><p>In this article, we will explain how we integrated SSO into our platform. Our approach was based on <a href="https://support.zendesk.com/hc/en-us/articles/4408845838874-Enabling-JWT-single-sign-on">this official Zendesk article</a>.</p><h3>Introduction</h3><p>We first need to understand how the login flow with a third-party service works within Zendesk. In this case, the first step is to <a href="https://support.zendesk.com/hc/en-us/articles/4408845838874-Enabling-JWT-single-sign-on#topic_gds_ydj_zj">create a new SSO client within Zendesk</a>. To do this, Zendesk will ask for the URL to which users should be redirected when they select this login option. After completing this setup, Zendesk will provide a key that we must use to sign the tokens that will authenticate users within the Zendesk platform.</p><p>Once we have the key to sign the Zendesk login token, we’ll go through the detailed steps needed t<a href="https://support.zendesk.com/hc/en-us/articles/4408845838874-Enabling-JWT-single-sign-on#topic_xml_kdj_zj">o authenticate a user from our platform in Zendesk</a>:</p><ol><li>A user wants to log in to our Zendesk through our platform</li><li>Zendesk redirects this user to the URL with the path that we indicate in the configuration</li><li>The route within our platform to which Zendesk redirects our user validates whether the user is already logged in within our platform or forces them to log in.</li><li>Once the user has an active session within JOOR, we ask our API to generate a session token for Zendesk using the key that Zendesk provided us.</li><li>Our web portal redirects with this session token to Zendesk</li><li>The user finally accesses Zendesk as a logged-in user</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ZgGkvzPUjsLKoIQC_Plzw.png" /><figcaption>Login process</figcaption></figure><h3>Implementation details</h3><p>Once we have a clear understanding of the generic process for authenticating users in Zendesk from a third-party service, we can explain how we’ve implemented it within our platform.</p><p>The JOOR platform can be summarized as a Single Page Application (SPA) built with React, along with an API that retrieves the data clients see on each page. Both our web application and API have certain endpoints that are public (access without being logged in) and others that are private (logged-in access only).</p><p>To implement this authentication flow within our platform, we need to create a private endpoint in our API that logged-in users can use to request their Zendesk authentication token. This endpoint must be private and accessible only to registered users. It will return a JWT signed with the secret provided by Zendesk. The structure of the JWT expected by Zendesk is described <a href="https://support.zendesk.com/hc/en-us/articles/4408845838874-Enabling-JWT-single-sign-on#topic_otw_jfh_3fb">here</a>, and an example of the code would be:</p><pre>import jwt<br>import time<br>import uuid<br>import os<br><br>def generate_zendesk_sso_token(logged_user: User) -&gt; str:<br><br>  payload = {<br>      &#39;iat&#39;: int(time.time()),<br>      &#39;jti&#39;: str(uuid.uuid4()),<br>      &#39;external_id&#39;: logged_user.id,<br>      &#39;email&#39;: logged_user.email,<br>      &#39;email_verified&#39;: True,<br>      &#39;name&#39;: logged_user.name,<br>  }<br><br>  zendesk_secret = os.getenv(&#39;ZENDESK_SSO_SECRET&#39;)  <br>  token = jwt.encode(payload, zendesk_secret, algorithm=&#39;HS256&#39;)<br><br>  return token<br></pre><p>Once we have this API endpoint implemented, we move on to our SPA page. In this case, it will be a page on a new path that simply redirects to Zendesk via a POST request. Before performing this POST redirection, it will need to obtain the Zendesk token from the API to authenticate the user.</p><pre>$ # cURL format required by Zendesk<br>$ curl --location &#39;https://&lt;your-zendesk-domain&gt;.zendesk.com/access/jwt&#39; \<br>--header &#39;Content-Type: application/x-www-form-urlencoded&#39; \<br>--data-urlencode &#39;jwt=&lt;user-jwt&gt;&#39; \<br>--data-urlencode &#39;return_to=&lt;return_to_url&gt;&#39;</pre><p>In the official Zendesk article we’ve mentioned several times here, they explain that when Zendesk redirects users for login, they add a query parameter called return_to, which indicates the original path from which our users decided to log in. To improve their browsing experience, it’s highly recommended to send this parameter back to Zendesk, so these users can continue navigating from the point where they left off. With that in mind, our React component would look like this:</p><pre>const ZendeskSSOLoginPage = () =&gt; {<br>  const DEFAULT_ZENDESK_URL = getDefaultZendeskUrl()<br>  const ZENDESK_SSO_URL = getZendeskSSOUrl()<br><br>  const { data, loading } = useFetchSSOToken()<br>  const searchParams = new URLSearchParams(useLocation().search)<br>  const returnTo = searchParams.get(&#39;return_to&#39;) || DEFAULT_ZENDESK_URL<br><br>  const sendPostRequest = (jwt_token: string, returnTo: string) =&gt; {<br>    const form = document.createElement(&#39;form&#39;)<br>    form.method = &#39;POST&#39;<br>    form.action = ZENDESK_SSO_URL<br><br>    const jwtInput = document.createElement(&#39;input&#39;)<br>    jwtInput.type = &#39;hidden&#39;<br>    jwtInput.name = &#39;jwt&#39;<br>    jwtInput.value = encodeURIComponent(jwt_token)<br>    form.appendChild(jwtInput)<br><br>    const returnToInput = document.createElement(&#39;input&#39;)<br>    returnToInput.type = &#39;hidden&#39;<br>    returnToInput.name = &#39;return_to&#39;<br>    returnToInput.value = encodeURIComponent(returnTo)<br>    form.appendChild(returnToInput)<br><br>    document.body.appendChild(form)<br>    form.submit()<br>  }<br>  useEffect(() =&gt; {<br>    if (!loading &amp;&amp; !!data) {<br>      sendPostRequest(data.token, returnTo)<br>    }<br>  }, [data, loading])<br><br><br>  return (<br>    &lt;LoaderComponent&gt;<br>        Redirecting...<br>    &lt;/LoaderComponent&gt;<br>  )<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*5NdmrJjkRHniQybqZEkr8w.png" /><figcaption>JOOR Zendesk SSO integration workflow</figcaption></figure><h3>Conclusion</h3><p>As seen in the article, integrating our platform with Zendesk to authenticate users has been quite straightforward. If you have any questions or suggestions, please let us know. We hope this article has been helpful to you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4394aeef89d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/joor-engineering/how-we-integrate-zendesk-sso-login-into-joor-4394aeef89d2">How we integrate Zendesk SSO login into JOOR</a> was originally published in <a href="https://medium.com/joor-engineering">JOOR Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting Up a Local AWS S3 Environment for Development: A Step-by-Step Guide]]></title>
            <link>https://medium.com/joor-engineering/setting-up-a-local-aws-s3-environment-for-development-a-step-by-step-guide-573353200d32?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/573353200d32</guid>
            <category><![CDATA[localstack]]></category>
            <category><![CDATA[devex]]></category>
            <category><![CDATA[s3]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[dev]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Tue, 09 Apr 2024 08:33:11 GMT</pubDate>
            <atom:updated>2024-04-09T08:50:17.266Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>These days, it’s very common for Software Engineers to create applications that require access to third-party services. Examples of such third-party services include APIs, databases, and cloud-based applications. Many applications and companies operate and deploy their services on Amazon Web Services (AWS). Additionally, one of the first and most renowned services offered by AWS is Simple Storage Service (S3).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oauNPPrSqH6YKQPcLb6Kzg.jpeg" /></figure><p>When Docker entered the realm of development, setting up a local database for development became extremely easy. You could create a docker-compose.yml file where you had both the database and your service defined. This allowed for relatively simple end-to-end testing of certain services locally.</p><p>But what about other third-party services? Do I have to create mocks in my tests? Furthermore, in the case of certain services like S3, operations (both read and write) incur a cost. Would I have to pay every time I want to perform an end-to-end test?</p><p>Each type of third-party service has its way of being simulated. In this case, I’m here to explain how we can simulate S3 locally, enabling us to carry out all the operations that S3 allows without needing to access the AWS cloud.</p><h3>LocalStack</h3><p><a href="https://www.localstack.cloud/">LocalStack</a> is a powerful tool used in software development for creating local environments that replicate various AWS cloud services. It allows developers to emulate the behavior of AWS services, such as S3 (Simple Storage Service), SQS (Simple Queue Service), SNS (Simple Notification Service), DynamoDB (NoSQL database), and many others, on their local machines.</p><p>Essentially, LocalStack provides a local sandbox environment that mimics the AWS cloud infrastructure, enabling developers to test their applications locally without incurring any costs associated with using the actual AWS services. This is particularly useful for development and testing purposes, as it allows developers to experiment with AWS services, develop and debug their applications, and run end-to-end tests without relying on the internet or incurring expenses.</p><p>Moreover, this tool comes neatly packaged in a <a href="https://hub.docker.com/r/localstack/localstack">Docker image</a>, ready to be configured in our typical docker-compose.yml file.</p><p>In this article, we won’t delve deeply into all the options and what can or cannot be done, as LocalStack has very good documentation on its website.</p><h3>Example</h3><p>In this section, we’re going to perform an exercise as generic and simple as possible. For this reason, we’ll create a playground environment with the following requirements:</p><ul><li>We’ll have two containers: a Debian and a LocalStack.</li><li>When we start the playground environment (docker compose up -d), the S3 bucket named ‘<em>david-garcia-medium</em>’ must exist.</li><li>From the Debian container, using AWS CLI, I should be able to perform operations with S3 as if it were the actual AWS itself.</li></ul><p>First of all, let’s display the file structure of our repository.</p><pre>./<br>  playground/<br>    aws/<br>      config<br>      credentials<br>    aws_cli/<br>      awscliv2-amd64.zip<br>      awscliv2-arm64.zip<br>    localstack/<br>      s3.sh<br>  .env.debian<br>  .env.localstack<br>  docker-compose.yml<br>  Dockerfile</pre><p>In the ‘playground’ directory, we have the necessary configuration to make our playground environment work. In this case, we have three subdirectories:</p><ul><li><strong>aws</strong>: which contains the authentication configuration required by AWS CLI to access AWS.</li><li><strong>aws_cli</strong>: where we have the downloaded AWS CLI binaries — in this case, we’ve downloaded both the AMD (for Linux in general) and ARM (for Mac with M1 in general) versions.</li><li><strong>localstack</strong>: with a script, which will be used to configure the startup of the LocalStack container.</li></ul><p>The .env.debian and .env.localstack files are files with declared environment variables that will be mounted to the containers in the docker-compose.yml. They contain the following information</p><pre># .env.debian<br># AWS cli configuration<br>AWS_PROFILE=localstack</pre><p>The .env.debian is very simple, and only contains <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-using-profiles">the AWS environment variable</a> used by AWS CLI to know what profile must be used by default. If we review the files available in playground/aws path we find the following</p><pre># config<br>[profile localstack]<br>output = json<br>endpoint_url = http://localstack:4566<br>region = us-east-1</pre><pre># credentials<br>[localstack]<br>aws_access_key_id=test<br>aws_secret_access_key=test</pre><p>In the playground/aws/config file, we overwrite the endpoint_url to use our LocalStack as AWS backend instead of the official AWS. In the playground/aws/credentials we configure the credentials to the localstack profile.</p><p>Let’s review the .env.localstack</p><pre># .env.localstack<br><br>SERVICES=s3<br><br># Checker<br>TRIES=30<br><br># Base<br>HOSTNAME_EXTERNAL=localstack<br><br>LOCALSTACK_API=http://localstack:4566<br>LOCALSTACK_HEALTH_ENDPOINT=http://localstack:4566/health<br><br># S3<br>BUCKET_NAMES=david-garcia-medium<br><br># AWS cli configuration<br>AWS_DEFAULT_PROFILE=localstack</pre><p>These environment variables are typically utilized by the script located at playground/localstack/s3.sh. This script serves as a fundamental tool that executes once the LocalStack container is up and running, facilitating the creation of the pre-existing S3 bucket that we require. This script will be mounted in the path /etc/localstack/initi/ready.d and the localstack container will execute when it is ready. More info <a href="https://docs.localstack.cloud/references/init-hooks/">here</a></p><p>Let’s review the docker image with debian, the Dockerfile</p><pre>FROM docker.io/debian:bookworm-slim<br><br># Added TARGETARCH to differenciate between amd64 (Linux and Windows) and arm64 (Mac) when install awscliv2.zip<br>ARG TARGETARCH<br><br># Install zip<br>RUN apt update &amp;&amp; \<br>    apt install -y zip=3.0-13 &amp;&amp; \<br>    rm -rf /var/lib/apt/lists/*<br><br># Install awscli<br>COPY playground/aws_cli/awscliv2-${TARGETARCH}.zip .<br>RUN unzip awscliv2-${TARGETARCH}.zip &amp;&amp; \<br>    ./aws/install &amp;&amp; \<br>    rm -rf awscliv2-${TARGETARCH}.zip aws<br><br># By default, this container does not execute anything; it simply sleeps indefinitely.<br>CMD [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]</pre><p>In this scenario, the image is quite straightforward, with the only sophisticated aspect being our readiness to build the image for both amd64 and arm64 architectures, utilizing the TARGETARCH Docker ARG. We use the binaries stored in playground/aws_cli instead of downloading them via curl in the Dockerfile to ensure consistent installation of the same version of AWS CLI. This practice guarantees that each time we rebuild the image, we maintain uniformity in the awscli version.</p><p>And finally, we see the docker-compose.yml file with the definition of the services before running a test and verifying that everything works.</p><pre>version: &#39;3.7&#39;<br><br><br>services:<br>  debian:<br>    build:<br>      context: .<br>      dockerfile: Dockerfile<br>    depends_on:<br>      - localstack<br>    env_file:<br>      - .env.debian<br>    volumes:<br>      - ./playground/aws:/root/.aws:ro<br><br><br>  localstack:<br>    image: docker.io/localstack/localstack:3.1<br>    ports:<br>      - &quot;4566:4566&quot;<br>    env_file:<br>      - .env.localstack<br>    volumes:<br>      - ./playground/localstack:/etc/localstack/init/ready.d/:ro<br>      - ./playground/aws:/root/.aws:ro</pre><p>As you can see, there’s nothing unfamiliar here that we haven’t commented on previously. Let’s test it:</p><pre>$ docker compose down -v &amp;&amp; docker compose up -d &amp;&amp; docker compose exec debian bash<br>...<br>root@9698486ac759:/# # List the aws s3 buckets availables<br>root@9698486ac759:/# aws s3 ls  <br>2024-04-02 17:57:15 david-garcia-medium<br>root@9698486ac759:/# # Create a file and upload to S3<br>root@9698486ac759:/# echo &quot;hello medium&quot; &gt; my_file.txt<br>root@9698486ac759:/# aws s3 cp my_file.txt s3://david-garcia-medium<br>upload: ./my_file.txt to s3://david-garcia-medium/my_file.txt   <br>root@9698486ac759:/# aws s3 ls s3://david-garcia-medium<br>2024-04-02 18:02:06         13 my_file.txt<br>root@9698486ac759:/# # Create a new S3 bucket<br>root@9698486ac759:/# aws s3 mb &quot;s3://my-new-bucket&quot;<br>make_bucket: my-new-bucket<br>root@9698486ac759:/# aws s3 ls                     <br>2024-04-02 17:57:15 david-garcia-medium<br>2024-04-02 18:04:20 my-new-bucket<br>root@9698486ac759:/# # you can test other commands!</pre><p><strong>Extra example</strong></p><p>Furthermore, LocalStack provides the files we have uploaded to S3 at the following path: http://localhost:4566/&lt;bucket-name&gt;, and you can download them using the S3 key.</p><pre>$ curl http://localhost:4566/david-garcia-medium<br>&lt;ListBucketResult xmlns=&quot;http://s3.amazonaws.com/doc/2006-03-01/&quot;&gt;<br>  &lt;IsTruncated&gt;false&lt;/IsTruncated&gt;<br>  &lt;Marker/&gt;<br>  &lt;Name&gt;david-garcia-medium&lt;/Name&gt;<br>  &lt;Prefix/&gt;<br>  &lt;MaxKeys&gt;1000&lt;/MaxKeys&gt;<br>  &lt;Contents&gt;<br>    &lt;Key&gt;my_file.txt&lt;/Key&gt;<br>    &lt;ETag&gt;&quot;5047cdd6613d55aa1a3143639a81cf78&quot;&lt;/ETag&gt;<br>    &lt;Owner&gt;<br>      &lt;DisplayName&gt;webfile&lt;/DisplayName&gt;<br>      &lt;ID&gt;75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a&lt;/ID&gt;<br>    &lt;/Owner&gt;<br>    &lt;Size&gt;13&lt;/Size&gt;<br>    &lt;LastModified&gt;2024-04-02T18:02:06.000Z&lt;/LastModified&gt;<br>    &lt;StorageClass&gt;STANDARD&lt;/StorageClass&gt;<br>  &lt;/Contents&gt;<br>&lt;/ListBucketResult&gt;<br><br>$ curl http://localhost:4566/david-garcia-medium/my_file.txt<br>hello medium</pre><p>To conclude, you have all the code available in the following <a href="https://github.com/own3dh2so4/localstack_s3_example">GitHub repository</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=573353200d32" width="1" height="1" alt=""><hr><p><a href="https://medium.com/joor-engineering/setting-up-a-local-aws-s3-environment-for-development-a-step-by-step-guide-573353200d32">Setting Up a Local AWS S3 Environment for Development: A Step-by-Step Guide</a> was originally published in <a href="https://medium.com/joor-engineering">JOOR Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FOSDEM 2024 My Review]]></title>
            <link>https://medium.com/joor-engineering/fosdem-2024-my-review-d7a4c7f3c547?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7a4c7f3c547</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[fosdem]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Tue, 20 Feb 2024 11:20:35 GMT</pubDate>
            <atom:updated>2024-02-20T11:20:35.138Z</atom:updated>
            <content:encoded><![CDATA[<p>My name is David Garcia, staff engineer at JOOR, and a few weeks ago I was lucky to attend my second FOSDEM, let’s see what I found there …</p><h3>Why FOSDEM?</h3><p>For me, it’s an incredibly exciting experience to witness the latest developments in open-source projects and to learn about new technologies that have emerged over the past year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r1INWN4t79Orn1bGTWno9Q.png" /><figcaption>Image from FOSDEM 2024 Highlights 20:13</figcaption></figure><p>I’m a backend engineer with a strong interest in databases, observability, Linux, and programming. In my current role, I work extensively with Python, PostgreSQL, and Kubernetes. Therefore, the most interesting tracks for me were:</p><ul><li><a href="https://fosdem.org/2024/schedule/track/containers/">Containers devroom</a>: Saturday 3 February 2024</li><li><a href="https://fosdem.org/2024/schedule/track/monitoring-observability/">Monitoring &amp; Observability devroom</a>: Sunday 4 February 2024</li><li><a href="https://fosdem.org/2024/schedule/track/postgresql/">PostgreSQL devroom</a>: Sunday 4 February 2024</li><li><a href="https://fosdem.org/2024/schedule/track/python-devroom/">Python Devroom devroom</a>: Sunday 4 February 2024</li></ul><h3>How to prepare for it?</h3><p>I always start by selecting my favourite tracks to choose the talks that I find most interesting there. Once I’ve chosen the talks I’m most interested in from my favourite tracks, I then explore the remaining tracks to see which talks I like.</p><p>An important thing to keep in mind about FOSDEM is that not all rooms where talks are held are equally large, nor are they all in the same building. It’s worth considering that if a talk looks very promising, it may fill up before its scheduled time, and you might end up being unable to attend. Similarly, if it’s in a distant building, you may not make it in time.</p><p>To organise which talks to attend and to know where they are, my recommendation is to use the <a href="https://play.google.com/store/apps/details?id=be.digitalia.fosdem&amp;hl=es_419&amp;gl=US">FOSDEM Companion</a> app. This app not only includes all the tracks with their talks and schedules, but it also provides maps of the university campus to show you where the buildings are located, and within each building, it indicates the location of each room.</p><h3>My 2024 favourite talks</h3><p>With that said, here are my favourite talks:</p><ul><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-1855-forensic-container-checkpointing-and-analysis/">Forensic container checkpointing and analysis (Containers devroom)</a>: a very interesting talk about checkpointing containers. It’s a continuation of <a href="https://archive.fosdem.org/2023/schedule/event/container_kubernetes_criu/">the talk</a> he gave last year.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2988-introducing-incus/">Introducing Incus (Containers devroom)</a>: It teaches us what Icus (fully-fledged system container and virtual machine manager) is, in addition to providing a live demo, which is always appreciated.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3595-open-food-facts-acting-on-the-health-and-environnemental-impacts-of-the-food-system/">Open Food Facts : Acting on the health and environmental impacts of the food system (Lightning talks)</a>: It shows us how, through an OpenSource community, we can monitor the food we buy at the supermarket.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3669-reinventing-database-exploration-with-azimutt/">Reinventing database exploration with Azimutt (Lightning talks)</a>: In this case, since I’ve been reviewing a very large database in my last month of work, this type of tool would have been very helpful for me.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-2181-passbolt-open-source-password-manager-for-teams/">Passbolt — Open source password manager for teams (Lightning talks)</a>: It teaches us how to have our own password manager, in addition to giving us a sneak peek at some features that will come in the future.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3747-testing-containers-with-python-and-pytest/">Testing Containers with Python and pytest (Lightning talks)</a>: It teaches us about the <a href="https://pypi.org/project/pytest-container/">pytest-container</a> library, a library that I wasn’t familiar with and found super interesting and useful for testing our Docker images.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3543-platform-engineering-for-dummies/">Platform engineering for dummies (Lightning talks)</a>: What is platform engineering? Here you have the detailed answer.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3600-isolation-levels-and-mvcc-in-sql-databases-a-technical-comparative-study/">Isolation Levels and MVCC in SQL Databases: A Technical Comparative Study (PostgreSQL devroom)</a>: A graphical and visual explanation of the different types of isolation levels in database transactions and how PostgreSQL applies them.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3601-reducing-costs-and-improving-performance-with-data-modeling-in-postgres/">Reducing Costs and Improving Performance With Data Modeling in Postgres (PostgreSQL devroom)</a>: It explains how data is stored on disk in PostgreSQL and how the order in which we declare columns in a table matters and can help us save space.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-1921-linux-load-average-and-other-silly-metrics/">Linux load average and other silly metrics (Monitoring &amp; Observability devroom)</a>: It teaches us how to interpret typical metrics of Linux systems, such as load average.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3263-practical-ci-cd-observability-with-opentelemetry/">Practical CI/CD Observability with OpenTelemetry (Monitoring &amp; Observability devroom)</a>: A good practical example of integration with OpenTelemetry.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3499-implementing-distributed-traces-with-ebpf/">Implementing distributed traces with eBPF (Monitoring &amp; Observability devroom)</a>: In this case, they teach us how to leverage the features provided by eBPF to add traces to our applications in order to monitor the entire system.</li><li><a href="https://fosdem.org/2024/schedule/event/fosdem-2024-3604-your-virtual-dba-postgresql-on-kubernetes-using-an-operator-/">Your Virtual DBA (PostgreSQL on Kubernetes using an Operator) (PostgreSQL devroom)</a>: They discuss the role of a DBA and the requirements a Kubernetes Operator must have to deploy a database.</li></ul><p>These are the talks I’ve liked the most so far. I haven’t finished watching all the ones I couldn’t attend in person yet. Additionally, some of them haven’t been uploaded to the website yet, so I still have to wait a bit.</p><p>This year has left me with a good impression, and to be honest, I’m looking forward to attending again next year.</p><h3>About JOOR</h3><p>Founded in 2010, JOOR now services more than 14,000 brands and over 600,000 buyers across 150 countries, processing an incredible $1.7B in wholesale transaction volume each month.</p><p>We have some open positions from time to time, if interested and curious just check them <a href="https://www.joor.com/current-openings">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7a4c7f3c547" width="1" height="1" alt=""><hr><p><a href="https://medium.com/joor-engineering/fosdem-2024-my-review-d7a4c7f3c547">FOSDEM 2024 My Review</a> was originally published in <a href="https://medium.com/joor-engineering">JOOR Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Raspberry Pi: Install Docker and set up a public website with Traefik, Let’s Encrypt, and DuckDNS.]]></title>
            <link>https://medium.com/@own3dh2so4/raspberry-pi-install-docker-and-set-up-a-public-website-with-traefik-lets-encrypt-and-duckdns-718ef3992699?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/718ef3992699</guid>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[traefik]]></category>
            <category><![CDATA[duckdns]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Mon, 19 Feb 2024 07:02:02 GMT</pubDate>
            <atom:updated>2024-02-19T07:02:02.931Z</atom:updated>
            <content:encoded><![CDATA[<p>In <a href="https://medium.com/@own3dh2so4/raspberry-pi-my-first-steps-936d03eb7f6d">my latest article</a> detailing my initial step with my Raspberry Pi, I wrapped up with the basic network configuration — sufficient for accessing it via SSH from my laptop.</p><p>In this article, I will discuss how to install Docker and set up a public website with HTTPS. In the past, this task was quite complex and often required a small expense. However, nowadays, thanks to <a href="https://letsencrypt.org/">Let’s Encrypt</a> for generating HTTPS certificates and <a href="https://www.duckdns.org/">DuckDNS</a> for providing domain names ending in duckdns.org, this process has become much simpler.</p><h4><strong>Installing docker</strong></h4><p>Currently, when I install Docker, I use the official script available <a href="https://get.docker.com/">here</a>.</p><pre>$ curl -fsSL https://get.docker.com -o get-docker.sh<br># Review the script<br>$ sudo sh ./get-docker.sh</pre><blockquote>I always recommend reviewing the scripts thoroughly and avoiding direct installation by piping the curl result. Reason <a href="https://lukespademan.com/blog/the-dangers-of-curlbash/">here</a></blockquote><p>The newest versions of Docker come with Docker Compose already installed, accessible as a subcommand of Docker. Instead of using the command docker-compose &lt;sub-command&gt;, you should write docker compose &lt;subcommand&gt;.</p><p>I highly recommend configuring Docker for personal use to <a href="https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user">manage as a non-root user</a> and <a href="https://docs.docker.com/engine/install/linux-postinstall/#configure-docker-to-start-on-boot-with-systemd">setting it to start on boot</a>. After that we can test our installation</p><pre>$ docker -v<br>Docker version 25.0.1, build 29cf629<br>$ docker run --rm hello-world<br>Unable to find image &#39;hello-world:latest&#39; locally<br>latest: Pulling from library/hello-world<br>478afc919002: Pull complete <br>Digest: sha256:d000bc569937abbe195e20322a0bde6b2922d805332fd6d8a68b19f524b7d21d<br>Status: Downloaded newer image for hello-world:latest<br><br>Hello from Docker!<br>This message shows that your installation appears to be working correctly.<br><br>To generate this message, Docker took the following steps:<br> 1. The Docker client contacted the Docker daemon.<br> 2. The Docker daemon pulled the &quot;hello-world&quot; image from the Docker Hub.<br>    (arm64v8)<br> 3. The Docker daemon created a new container from that image which runs the<br>    executable that produces the output you are currently reading.<br> 4. The Docker daemon streamed that output to the Docker client, which sent it<br>    to your terminal.<br><br>To try something more ambitious, you can run an Ubuntu container with:<br> $ docker run -it ubuntu bash<br><br>Share images, automate workflows, and more with a free Docker ID:<br> https://hub.docker.com/<br><br>For more examples and ideas, visit:<br> https://docs.docker.com/get-started/<br>$ docker rmi hello-world<br>Untagged: hello-world:latest<br>Untagged: hello-world@sha256:d000bc569937abbe195e20322a0bde6b2922d805332fd6d8a68b19f524b7d21d<br>Deleted: sha256:ee301c921b8aadc002973b2e0c3da17d701dcd994b606769a7e6eaa100b81d44<br>Deleted: sha256:12660636fe55438cc3ae7424da7ac56e845cdb52493ff9cf949c47a7f57f8b43</pre><blockquote>Another recommendation is to <a href="https://dalwar23.com/how-to-fix-no-memory-limit-support-for-docker-in-raspberry-pi/">enable </a><a href="https://dalwar23.com/how-to-fix-no-memory-limit-support-for-docker-in-raspberry-pi/">cgroups support</a> to allow limiting the resources that a Docker image can use. In my case, I have multiple Docker images running on the same Raspberry Pi, and I want to limit the resources of each image.</blockquote><h4>Configure DuckDNS</h4><p>The first step is to create an account on DuckDNS, which is very simple and can be done using Single Sign-On (SSO) with your Google account. When you are logged in, you will see the following screen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SCXBaJ_HO1WxnYeAgF2t6g.png" /><figcaption>DuckDNS website</figcaption></figure><p>In the last image, I’ve hidden some personal information, but I’ve added two red arrows to highlight the most important sections. One arrow shows how to create a domain. It’s simple: enter your favorite domain name, and if it’s not already taken by another user, click on ‘add domain’.</p><p>The other arrow shows your personal token. This token is needed to update the IP associated with the domain you reserved.</p><h4>Create docker-compose.yml</h4><p>With our DuckDNS account created, it’s time to create a docker-compose.ymlfile for our public website. For this example, I will use the <a href="https://github.com/traefik/whoami">whoami</a> service, but you can use NGINX or any other service you prefer. Let’s examine the file and then we’ll explain it section by section.</p><pre>version: &quot;3.9&quot;<br><br>services:<br>  traefik:<br>    image: &quot;traefik:v2.11&quot;<br>    container_name: &quot;traefik&quot;<br>    restart: unless-stopped<br>    mem_limit: 30m<br>    command:<br>      - &quot;--api.insecure=true&quot;<br>      - &quot;--providers.docker=true&quot;<br>      - &quot;--providers.docker.exposedbydefault=false&quot;<br>      - &quot;--providers.docker.network=traefik&quot;<br>      - &quot;--entrypoints.web.address=:80&quot;<br>      - &quot;--entrypoints.websecure.address=:443&quot;<br>      - &quot;--certificatesresolvers.myresolver.acme.dnschallenge=true&quot;<br>      - &quot;--certificatesresolvers.myresolver.acme.dnschallenge.provider=duckdns&quot;<br>      - &quot;--certificatesresolvers.myresolver.acme.email=&lt;your email&gt;&quot;<br>      - &quot;--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json&quot;<br>    ports:<br>      - &quot;80:80&quot;<br>      - &quot;443:443&quot;<br>      - &quot;8080:8080&quot;<br>    environment:<br>      - DUCKDNS_TOKEN=${DUCKDNS_TOKEN}<br>    volumes:<br>      - &quot;./letsencrypt:/letsencrypt&quot;<br>      - &quot;/var/run/docker.sock:/var/run/docker.sock:ro&quot;<br>    networks:<br>     - &quot;default&quot;<br>     - &quot;traefik&quot;<br><br>  whoami:<br>    image: &quot;traefik/whoami&quot;<br>    container_name: &quot;whoami&quot;<br>    labels:<br>      - &quot;traefik.enable=true&quot;<br>      - &quot;traefik.http.routers.whoami.rule=Host(`whoami.&lt;your-domain&gt;.duckdns.org`)&quot;<br>      - &quot;traefik.http.routers.whoami.entrypoints=websecure&quot;<br>      - &quot;traefik.http.routers.whoami.tls.certresolver=myresolver&quot;<br>    mem_limit: 10m<br><br>networks:<br>  traefik:<br>    name: traefik</pre><p>In this docker-compose.yml file, we can see two services: whoami, the service that we’ve already discussed and that is the one we want to expose to the internet, and traefik, which is the main component and orchestrator.</p><blockquote>Traefik is a modern HTTP reverse proxy and load balancer that is designed to handle dynamic container environments. Traefik integrates seamlessly with Docker, Kubernetes, and other container orchestration platforms, making it easy to manage and scale applications with minimal configuration.</blockquote><p>Let’s review the Traefik command parameters listed in the docker-compose.yml file:</p><ul><li>--api.insecure=true: Enables the insecure API, allowing access to Traefik’s API without authentication. This is useful for development purposes but should be disabled in production environments.</li><li>--providers.docker=true: Enables Docker as a provider for dynamically discovering services.</li><li>--providers.docker.exposedbydefault=false: Configures Traefik to only expose services explicitly defined in Docker labels, rather than exposing all containers by default.</li><li>--providers.docker.network=traefik: Specifies the Docker network named traefik to use for communication between Traefik and Docker containers. I created this network at the end of the file ( networks: section) so that it can be used by other docker-compose.yml files on the same host.</li><li>--entrypoints.web.address=:80: Defines an entry point named “web” on port 80 for handling HTTP traffic.</li><li>-entrypoints.websecure.address=:443: Defines an entry point named “websecure” on port 443 for handling HTTPS traffic.</li><li>--certificatesresolvers.myresolver.acme.dnschallenge=true : Enables the ACME DNS-01 challenge for obtaining TLS certificates automatically from Let’s Encrypt.</li><li>--certificatesresolvers.myresolver.acme.dnschallenge.providers=duckdns: Specifies DuckDNS as the DNS provider for the ACME DNS challenge.</li><li>--certificatesresolvers.myresolver.acme.email=&lt;your-email&gt;: Sets the email address to be used for Let’s Encrypt certificate registration.</li><li>--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json: Defines the location to store ACME certificates obtained from Let’s Encrypt.</li></ul><p>In the ports: section, we expose port 80 for the web entry point ( — entrypoints.web.address=:80), port 443 for the websecure entry point ( — entrypoints.websecure.address=:80), and port 8080 for the Traefik management website.</p><p>In the environment: section, we added our DuckDNS token. With that, Traefik automatically calls DuckDNS to update the IP of our domain.</p><p>In the volumes: section, I add a volume to store the Let’s Encrypt configuration on our host rather than within the container. This is helpful in case we need to restart the container. The other volume is for the Docker socket. Traefik requires access to the Docker socket to monitor the containers running on our host and discover the labels and configurations.</p><p>The last section of the traefik container is networks:, where we configure both the default network and the traefik network. The traefiknetwork will be utilized by services outside of this docker-compose.yml file to be accessible by this traefik container.</p><p>The configuration for the whoami container is very straightforward. In this case, I’m configuring Traefik via Docker labels. The labels that I’ve added to this service are:</p><ul><li>traefik.enable=true: This label enables Traefik to manage routing for this service.</li><li>traefik.http.routers.whoami.rule=Host(`whoami.&lt;your-domain&gt;.duckdns.org`): This label specifies the rule for routing requests to this service. It instructs Traefik to route requests with the specified hostname to this service.</li><li>traefik.http.routers.whoami.entrypoints=websecure: This label defines the entry point through which incoming requests should be routed to this service.</li><li>traefik.http.routers.whoami.tls.certresolver=myresolver: This label configures Traefik to use the myresolver certificate resolver for obtaining TLS certificates for this service.</li></ul><h4>Start and test</h4><p>To initiate the services declared in this docker-compose.yml, we simply need to run docker comopose up -d. Once the containers are up and running (this might take a few minutes), we can navigate to the browser using the domain we’ve defined and check the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m89sb9OkgOeMk9Z7KM12Eg.png" /><figcaption>whoami exposed with a public domain and HTTPS</figcaption></figure><p>That’s all; with that, we conclude the exercise. Just like we’ve exposed this service, we can expose others, such as WordPress or anything else you desire.</p><h4>Disclaimers</h4><p>To perform this exercise, you must have a public IP address. However, some telecom companies have their customers behind a <a href="https://en.wikipedia.org/wiki/Carrier-grade_NAT">CGNAT</a> (Carrier-Grade NAT). If you are behind a CGNAT, you cannot do this.</p><p>Additionally, to access services that you have in your local network via the Internet, you need to configure your router for <a href="https://en.wikipedia.org/wiki/Port_forwarding">port forwarding</a>. Port forwarding redirects traffic from a port on your router to the port and host that you select. Each router has its own configuration method, but in general, it’s straightforward.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=718ef3992699" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Raspberry Pi: My first steps]]></title>
            <link>https://medium.com/@own3dh2so4/raspberry-pi-my-first-steps-936d03eb7f6d?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/936d03eb7f6d</guid>
            <category><![CDATA[basics]]></category>
            <category><![CDATA[raspberry-pi-4]]></category>
            <category><![CDATA[raspberry-pi-os]]></category>
            <category><![CDATA[raspberry-pi]]></category>
            <category><![CDATA[configuration]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Fri, 26 Jan 2024 14:28:19 GMT</pubDate>
            <atom:updated>2024-01-26T14:28:19.131Z</atom:updated>
            <content:encoded><![CDATA[<p>Similar to many software engineers, I have a keen curiosity, and one of the aspects I particularly enjoy in my role is dealing with infrastructure and systems, even though my job responsibilities are consistently close to software development.</p><p>For me, <a href="https://www.raspberrypi.com/">Raspberry Pi</a> provides an accessible, reliable, and cost-effective platform for experimenting with new technologies. Equally important, it enables me to keep these experiments running 24/7, a crucial aspect of understanding the software lifecycle.</p><p>For this reason, I always keep one or more Raspberry Pis up and running at home, each equipped with various tools as part of my hobby. In this post, I’ll provide a brief introduction to how I started with a Raspberry Pi and the essential software I installed on it.</p><h4>Getting a Raspberry Pi</h4><p>The initial step is to acquire a Raspberry Pi. Generally, I prefer the “pre-configured” bundles available in typical online stores like Amazon. These bundles typically include the Raspberry Pi, a power adapter, a microSD card, a case, and a fan.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ShbehepH44GwO6H5gf_MA.png" /><figcaption>Here are two of my Raspberry Pis: Raspberry Pi 2B on the left and Raspberry Pi 4B on the right.</figcaption></figure><h4>Choose the Operative System</h4><p>This is one of the most important steps, and it’s where you need to determine the intended use for your Raspberry Pi. There are various operating systems available for Raspberry Pi, with the majority being Linux-based, but you can also install Windows if you prefer!</p><p>In my case, I use the Raspberry Pi as a server to run applications. Therefore, I always opt for an OS without a graphical user interface (GUI). GUI consumes storage space and resources for a purpose I don’t utilize. That’s why I choose not to install it, but I highly recommend installing it if you are just starting out or if it’s your first Raspberry Pi.</p><p>In this section, I want to mention another use for the Raspberry Pi that works very well and was a pleasant surprise for me: creating a retro-console by emulating. I won’t delve into it much in this post, but I highly recommend checking out OS options like <a href="https://www.recalbox.com/">Recalbox</a>.</p><p>In this case, I chose the well-known <a href="https://www.raspberrypi.com/software/">Raspberry Pi OS</a> (old Raspbian). Over the years, I have installed other OSes like Arch, Ubuntu, and Manjaro, but the main reason for trying these different operating systems was to experiment and, in the case of Arch, to learn more about Linux. However, when my goal isn’t to play or learn about the OS itself, I always opt for Raspberry Pi OS. Why? Because it’s the official OS and has a larger community.</p><h4>Install the Operative System</h4><p>The current method of installing OSes on a Raspberry Pi is straightforward and user-friendly (older methods like NOOBS are less common now). To install Raspberry Pi OS, you can use a package called <em>rpi-imager</em>, available for Ubuntu, Windows, and MAC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*USGHO9zhCQK5qBj4dwmBDA.png" /><figcaption>RPI Imager GUI</figcaption></figure><p>As you can see in the image, the process is composed of three steps:</p><ol><li><em>Choose the OS</em>: They provide the most common options, but you can choose a custom .img if you want.</li><li><em>Choose the storage</em>: Select the microSD where you want to install it.</li><li><em>Write</em>: Initiate the process of writing the selected OS to the microSD.</li></ol><h4>The first boot-up</h4><p>The new versions of the OS, during their first boot-up, auto-configure and detect the hardware. The first interaction we have with the OS is when it prompts us for the desired username and password. This user will have sudo privileges and will be assigned to UID 1000 and GID 1000.</p><pre>$ id<br>uid=1000(david) gid=1000(david) groups=1000(david),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),102(input),105(render),106(netdev),993(gpio),994(i2c),995(spi)</pre><blockquote>Don’t forget this username and password; without this information, you cannot log in to the Raspberry Pi, and you will need to format the microSD card again.</blockquote><h4>Keyboard Layout configuration</h4><p>In my case, being a non-English speaker (I am from Spain), the default <a href="https://en.wikipedia.org/wiki/List_of_QWERTY_keyboard_language_variants">keyboard layout</a> may not match my current layout. Therefore, my first step is to update it.</p><p>There are multiple ways to update it, but I always use the following command:</p><pre>$ sudo dpkg-reconfigure keyboard-configuration</pre><p>This command opens an interactive window that allows you to choose the correct layout.</p><h4>Network configuration</h4><p>When setting up the network, you can choose between a wired or wireless connection. In the case of a wired connection, the Raspberry Pi will typically obtain an IP address automatically from the router through DHCP. For a wireless connection, you will need to select your Wi-Fi network and enter the password.</p><p>The most modern Raspberry Pi includes a WiFi adapter. For this reason, I will describe how to connect to a WiFi network with it. In this area, the user experience has also improved significantly, thanks to the <em>nmcli</em> command.</p><pre>$ # FIRST STEP: ENABLE YOUR Wi-Fi Device<br>$ # Describe the status of the WiFi card<br>$ nmcli dev status<br>DEVICE         TYPE      STATE                   CONNECTION    <br>wlan0          wifi      disconnected            --<br>lo             loopback  connected (externally)  lo            <br>p2p-dev-wlan0  wifi-p2p  disconnected            --            <br>eth0           ethernet  unavailable             --    <br>$ # We can see that wifi is disconnect<br>$ # If you want to know if the wifi card is enabled execute the follow command<br>$ nmcli radio wifi<br>enabled<br>$ # In this case is enabled, if you have it disable you can execute <br>$ # the following command to enable it<br>$ nmcli radio wifi on<br>$ # SECOND STEP: IDENTIFY A Wi-Fi ACCESS POINT<br>$ # Scanning for nearby Wi-Fi networks<br>$ nmcli dev wifi list<br>IN-USE  BSSID              SSID                              MODE   CHAN  RATE        SIGNAL  BARS  SECURITY    <br>        78:29:ED:2E:0B:E9  MOVISTAR_0BE8                     Infra  6     130 Mbit/s  89      ▂▄▆█  WPA2        <br>        78:29:ED:2E:0B:F6  MOVISTAR_PLUS_0BE8                Infra  100   540 Mbit/s  85      ▂▄▆█  WPA2        <br>        66:29:ED:2E:0B:F6  MOVISTAR_0BE8                     Infra  100   540 Mbit/s  85      ▂▄▆█  WPA2        <br>        10:50:72:8E:32:81  SercommBB5005                     Infra  13    195 Mbit/s  72      ▂▄▆_  WPA2        <br>        10:50:72:8E:32:85  SercommBB5005_5G                  Infra  52    540 Mbit/s  64      ▂▄▆_  WPA2        <br>        E4:75:DC:A8:EF:D6  MiFibra-EFD4                      Infra  1     130 Mbit/s  59      ▂▄▆_  WPA2        <br>        58:B5:68:12:1E:28  1hxVGu7rkIWlakYrR2ckkr9mOZBqrUzj  Infra  11    0 Mbit/s    55      ▂▄__  WPA2 802.1X <br>        B2:46:FC:8B:DD:90  MOVISTAR_DD90                     Infra  6     130 Mbit/s  54      ▂▄__  WPA2 <br>$ # THIRD STEP: CONNECT TO Wi-Fi<br>$ sudo nmcli dev wifi connect &lt;network-ssid&gt; password &lt;network-password&gt;<br>$ # Example that we want connect to &quot;MOVISTAR_0BE8&quot; with password &quot;MyS3cr3tP4ssw0rd!&quot;<br>$ sudo nmcli dev wifi connect MOVISTAR_0BE8 password MyS3cr3tP4ssw0rd!</pre><p>Another important aspect for me is configuring a static IP for my Raspberry Pi within my private network. I do this to prevent IP changes in case the Raspberry Pi is rebooted or loses connections temporarily. In this scenario, I prefer configuring the static IP on my router, as it is responsible for managing IPs in my network through DHCP and can reserve certain IPs. While it’s possible to perform this task on the Raspberry Pi itself, doing so can lead to issues, and the configuration would be scattered across multiple devices instead of being centralized in one place: the router.</p><blockquote>Tip: You can assign a name to your Raspberry Pi, allowing you to connect to it using the name instead of the IP address. For example, you can edit your <em>/etc/hosts</em> file on Linux to include this mapping.</blockquote><h4><strong>Enable sshd</strong></h4><p>In this scenario, <em>sshd</em> is installed by default on all OS, but it’s disabled for security reasons. To enable it on Raspberry Pi OS, you can use <em>systemd</em>.</p><pre>$ # Enable sshd<br>$ sudo systemctl enable ssh<br>$ # Check if sshd is actived<br>$ sudo systemctl status ssh<br>● ssh.service - OpenBSD Secure Shell server<br>     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; preset: enabled)<br>     Active: active (running) since Fri 2024-01-26 10:34:50 GMT; 3h 39min ago<br>       Docs: man:sshd(8)<br>             man:sshd_config(5)<br>   Main PID: 657 (sshd)<br>      Tasks: 1 (limit: 8741)<br>        CPU: 840ms<br>     CGroup: /system.slice/ssh.service<br>             └─657 &quot;sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups&quot;</pre><blockquote>Tip: You can configure <em>sshd</em> to log in with a private certificate to avoid entering the username and password each time. Another security measure is to only allow logins with these certificates.</blockquote><h4>Conclusion</h4><p>With this, we have covered all the basics. After this, you can move your Raspberry Pi to another location and connect to it via SSH from your main PC. In my case, I have my Raspberry Pis behind the TV or on higher shelves. Every time I need to manage something on them, I connect to them via SSH.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=936d03eb7f6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Visualizing Database Dependencies: A Guide to Creating Dependency Graphs on Postgres]]></title>
            <link>https://medium.com/@own3dh2so4/visualizing-database-dependencies-a-guide-to-creating-dependency-graphs-on-postgres-52af01ac12fe?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/52af01ac12fe</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[graphviz]]></category>
            <category><![CDATA[postgresql]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Sat, 20 Jan 2024 20:18:20 GMT</pubDate>
            <atom:updated>2024-01-20T20:18:20.705Z</atom:updated>
            <content:encoded><![CDATA[<p>It is very common in the software industry for software engineers to begin working on a new project. In such cases, the initial steps that software engineers take involve searching for project documentation to understand its requirements. Unfortunately, in the majority of projects, the documentation is either missing, sparse, or outdated.</p><p>The next step is to delve into the code and the database schema. Depending on the size of the projects and the number of developers involved, understanding the complete requirements of the projects can be challenging at first glance. In this post, I will attempt to help you understand a database model by portraying it as a graph.</p><h4>Environment</h4><p>For this example, I will set up the following scenario in my local environment:</p><ul><li><strong>docker-compose.yml</strong>: containing a containerized <a href="https://hub.docker.com/_/postgres">PostgreSQL</a> and <a href="https://hub.docker.com/_/adminer/">Adminer</a></li><li><strong>schema.sql</strong>: defining a database schema created with ChatGPT</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4aa85a26947168731d570c248fb2571b/href">https://medium.com/media/4aa85a26947168731d570c248fb2571b/href</a></iframe><p>In this case, the script is available in the /docker-entrypoint-initdb.d directory inside the Docker PostgreSQL container and will be autogenerated during startup. For convenient database access, I’ve included Adminer, a web tool that allows you to connect to the database without any additional configuration on your local host.</p><h4>Understanding Postgres information_schema</h4><p><a href="https://www.postgresql.org/docs/current/information-schema.html"><em>information_schema</em> in PostgreSQL</a> is a set of views and tables that provide information about the structure and objects within the database. It serves as a metadata catalog, allowing users to query details about tables, columns, constraints, indexes, and other elements of the database. The most common views are:</p><ul><li>tables: Provides information about tables in the database, including details like table name, schema, type, and creation time.</li><li>columns: Contains information about columns of tables, including details such as column name, data type, and whether the column allows NULL values.</li><li>views: Offers information about views in the database, including details about the view name, schema, and the definition of the view.</li><li>table_constraints: Provides information about constraints on tables, including details about primary keys, unique constraints, foreign keys, etc.</li><li>key_column_usage: Contains information about columns that are used as keys in table constraints.</li><li>constraint_column_usage: Shows which columns are associated with which constraints.</li></ul><h4>How to programmatically create a graph?</h4><p>In this case, we can use <a href="https://graphviz.org/">Graphviz</a>, a graph visualization tool that facilitates the graphical representation of data structures. It utilizes a specific language called DOT to describe graph structures and automatically manages the layout of nodes and edges. This makes it convenient for creating visually appealing representations of relational or hierarchical data. Various libraries and websites support the creation of graphs in SVG format using Graphviz.</p><h4>Generate a graph of the database model</h4><p>It’s time to put all the explanations together. In this case, we will begin by adding all the tables from our schema to our graph.</p><pre>-- Get the tables of your schema in DOT language<br>SELECT &#39;&quot;&#39; || table_name || &#39;&quot;&#39;<br>FROM information_schema.tables <br>WHERE table_schema = &#39;public&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/211/1*I-X7jCMTVR7MddI32q1qrA.png" /><figcaption>Graph with the tables of our model</figcaption></figure><p>At this point, it doesn’t have much value; let’s add the references between them.</p><pre>-- Get the tables references between them in DOT language<br>SELECT<br>    &#39;&quot;&#39; || tc.table_name || &#39;&quot; -&gt; &quot;&#39; || ccu.table_name || &#39;&quot;;&#39;<br>FROM<br>    information_schema.table_constraints AS tc<br>JOIN information_schema.key_column_usage AS kcu ON<br>    tc.constraint_name = kcu.constraint_name<br>    AND tc.table_schema = kcu.table_schema<br>    AND tc.table_schema = &#39;public&#39;<br>JOIN information_schema.constraint_column_usage AS ccu ON<br>    ccu.constraint_name = tc.constraint_name<br>WHERE constraint_type = &#39;FOREIGN KEY&#39;;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*5bUU1NMVuk1GFAy5F_RH2A.png" /><figcaption>Graph image with tables and foreign keys</figcaption></figure><p>This image provides us with a lot more information. We can see that the “<em>users</em>” table is one of the core tables, as many other tables either directly or indirectly reference it. Additionally, we observe that the “<em>invoices</em>” domain is isolated, as its tables do not have relationships with the others.</p><p>What other iterations can we perform here? For example, separating the schema based on domains. In this case, we can easily identify the following domains: “<em>users</em>,” “<em>articles</em>,” “<em>invoices</em>,” “<em>task</em>,” and “<em>roles</em>”. To achieve this, we will represent each domain with a distinct color.</p><pre>-- Get the tables of your schema in DOT language with colors<br>SELECT &#39;&quot;&#39; || table_name || &#39;&quot; &#39; ||<br>CASE<br>  WHEN table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;[fillcolor=chartreuse3, style=filled];&#39;<br>  WHEN table_name LIKE &#39;article%&#39; <br>         OR table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;[fillcolor=orange, style=filled];&#39;<br>  WHEN table_name LIKE &#39;invoice%&#39;<br>         OR table_name = &#39;clients&#39; THEN &#39;[fillcolor=yellow, style=filled];&#39;<br>  WHEN table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;[fillcolor=blue, style=filled];&#39;<br>  WHEN table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;[fillcolor=deeppink, style=filled];&#39;<br>  ELSE &#39;;&#39;<br>END<br>FROM information_schema.tables <br>WHERE table_schema = &#39;public&#39;</pre><pre>-- Get the tables references between them in DOT language with colors<br>SELECT<br>    &#39;&quot;&#39; || tc.table_name || &#39;&quot;-&gt;&quot;&#39; || ccu.table_name || &#39;&quot; &#39;||<br>CASE<br>  WHEN tc.table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;[color=chartreuse3];&#39;<br>  WHEN tc.table_name LIKE &#39;article%&#39; <br>         OR tc.table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;[color=orange];&#39;<br>  WHEN tc.table_name LIKE &#39;invoice%&#39;<br>         OR tc.table_name = &#39;clients&#39; THEN &#39;[color=yellow];&#39;<br>  WHEN tc.table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;[color=blue];&#39;<br>  WHEN tc.table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;[color=deeppink];&#39;<br>  ELSE &#39;;&#39;<br>END<br>FROM<br>    information_schema.table_constraints AS tc<br>JOIN information_schema.key_column_usage AS kcu ON<br>    tc.constraint_name = kcu.constraint_name<br>    AND tc.table_schema = kcu.table_schema<br>    AND tc.table_schema = &#39;public&#39;<br>JOIN information_schema.constraint_column_usage AS ccu ON<br>    ccu.constraint_name = tc.constraint_name<br>WHERE constraint_type = &#39;FOREIGN KEY&#39;;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*MvkKrUK5iRDF-hLIG-jV5g.png" /><figcaption>Graph image with tables and foreign keys colored by domains</figcaption></figure><pre>/* This is the code generated by the last two queries */<br>digraph G {<br><br>    ranksep=1.0; size=&quot;18.5, 15.5&quot;; rankdir=LR;<br>    /* Tables in the schema with colors */<br>    &quot;invoices&quot; [fillcolor=yellow, style=filled];<br>    &quot;projects&quot; [fillcolor=blue, style=filled];<br>    &quot;tasks&quot; [fillcolor=blue, style=filled];<br>    &quot;users&quot; [fillcolor=chartreuse3, style=filled];<br>    &quot;comments&quot; [fillcolor=blue, style=filled];<br>    &quot;userroles&quot; [fillcolor=deeppink, style=filled];<br>    &quot;roles&quot; [fillcolor=deeppink, style=filled];<br>    &quot;clients&quot; [fillcolor=yellow, style=filled];<br>    &quot;invoicedetails&quot; [fillcolor=yellow, style=filled];<br>    &quot;tags&quot; [fillcolor=orange, style=filled];<br>    &quot;articles&quot; [fillcolor=orange, style=filled];<br>    &quot;articletags&quot; [fillcolor=orange, style=filled];<br>    &quot;images&quot; [fillcolor=orange, style=filled];<br>    &quot;articleimages&quot; [fillcolor=orange, style=filled];<br>    &quot;articlelikes&quot; [fillcolor=orange, style=filled];<br>    &quot;followers&quot; [fillcolor=chartreuse3, style=filled];<br>    &quot;articlecomments&quot; [fillcolor=orange, style=filled];<br>    /* Relations in the schema with colors */<br>    &quot;tasks&quot;-&gt;&quot;projects&quot; [color=blue];<br>    &quot;tasks&quot;-&gt;&quot;users&quot; [color=blue];<br>    &quot;comments&quot;-&gt;&quot;tasks&quot; [color=blue];<br>    &quot;comments&quot;-&gt;&quot;users&quot; [color=blue];<br>    &quot;userroles&quot;-&gt;&quot;users&quot; [color=deeppink];<br>    &quot;userroles&quot;-&gt;&quot;roles&quot; [color=deeppink];<br>    &quot;invoices&quot;-&gt;&quot;clients&quot; [color=yellow];<br>    &quot;invoicedetails&quot;-&gt;&quot;invoices&quot; [color=yellow];<br>    &quot;articles&quot;-&gt;&quot;users&quot; [color=orange];<br>    &quot;articletags&quot;-&gt;&quot;articles&quot; [color=orange];<br>    &quot;articletags&quot;-&gt;&quot;tags&quot; [color=orange];<br>    &quot;images&quot;-&gt;&quot;users&quot; [color=orange];<br>    &quot;articleimages&quot;-&gt;&quot;articles&quot; [color=orange];<br>    &quot;articleimages&quot;-&gt;&quot;images&quot; [color=orange];<br>    &quot;articlelikes&quot;-&gt;&quot;articles&quot; [color=orange];<br>    &quot;articlelikes&quot;-&gt;&quot;users&quot; [color=orange];<br>    &quot;followers&quot;-&gt;&quot;users&quot; [color=chartreuse3];<br>    &quot;followers&quot;-&gt;&quot;users&quot; [color=chartreuse3];<br>    &quot;articlecomments&quot;-&gt;&quot;articles&quot; [color=orange];<br>    &quot;articlecomments&quot;-&gt;&quot;users&quot; [color=orange];<br>}</pre><p>And the final step is to create the graph of your application domains and their dependencies.</p><pre>-- Get the domains of your schema in DOT language<br>SELECT distinct<br>CASE<br>  WHEN table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;&quot;users&quot; [fillcolor=chartreuse3, style=filled];&#39;<br>  WHEN table_name LIKE &#39;article%&#39; <br>         OR table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;&quot;articles&quot; [fillcolor=orange, style=filled];&#39;<br>  WHEN table_name LIKE &#39;invoice%&#39;<br>         OR table_name = &#39;clients&#39; THEN &#39;&quot;invoices&quot; [fillcolor=yellow, style=filled];&#39;<br>  WHEN table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;&quot;tasks&quot; [fillcolor=blue, style=filled];&#39;<br>  WHEN table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;&quot;roles&quot; [fillcolor=deeppink, style=filled];&#39;<br>  ELSE &#39;&quot;&#39; || table_name || &#39;&quot;;&#39;<br>END<br>FROM information_schema.tables <br>WHERE table_schema = &#39;public&#39;</pre><pre>-- Get the domains references between them in DOT language with colors<br>SELECT distinct<br>    &#39;&quot;&#39; ||<br>CASE<br>  WHEN tc.table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;users&#39;<br>  WHEN tc.table_name LIKE &#39;article%&#39; <br>         OR tc.table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;articles&#39;<br>  WHEN tc.table_name LIKE &#39;invoice%&#39;<br>         OR tc.table_name = &#39;clients&#39; THEN &#39;invoices&#39;<br>  WHEN tc.table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;tasks&#39;<br>  WHEN tc.table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;roles&#39;<br>  ELSE tc.table_name<br>END || &#39;&quot; -&gt; &quot;&#39; ||<br>CASE<br>  WHEN ccu.table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;users&#39;<br>  WHEN ccu.table_name LIKE &#39;article%&#39; <br>         OR ccu.table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;articles&#39;<br>  WHEN ccu.table_name LIKE &#39;invoice%&#39;<br>         OR ccu.table_name = &#39;clients&#39; THEN &#39;invoices&#39;<br>  WHEN ccu.table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;tasks&#39;<br>  WHEN ccu.table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;roles&#39;<br>  ELSE ccu.table_name<br>END || &#39;&quot; &#39; ||<br>CASE<br>  WHEN tc.table_name in (&#39;users&#39;, &#39;followers&#39;) THEN &#39;[color=chartreuse3];&#39;<br>  WHEN tc.table_name LIKE &#39;article%&#39; <br>         OR tc.table_name IN (&#39;tags&#39;, &#39;images&#39;) THEN &#39;[color=orange];&#39;<br>  WHEN tc.table_name LIKE &#39;invoice%&#39;<br>         OR tc.table_name = &#39;clients&#39; THEN &#39;[color=yellow];&#39;<br>  WHEN tc.table_name IN (&#39;tasks&#39;, &#39;comments&#39;, &#39;projects&#39;) THEN &#39;[color=blue];&#39;<br>  WHEN tc.table_name IN (&#39;userroles&#39;, &#39;roles&#39;) THEN &#39;[color=deeppink];&#39;<br>  ELSE &#39;;&#39;<br>END<br>FROM<br>    information_schema.table_constraints AS tc<br>JOIN information_schema.key_column_usage AS kcu ON<br>    tc.constraint_name = kcu.constraint_name<br>    AND tc.table_schema = kcu.table_schema<br>    AND tc.table_schema = &#39;public&#39;<br>JOIN information_schema.constraint_column_usage AS ccu ON<br>    ccu.constraint_name = tc.constraint_name<br>WHERE constraint_type = &#39;FOREIGN KEY&#39;;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*HVetm_j45NsvXB4cncysQQ.png" /><figcaption>Graph image with domains and relationship colored</figcaption></figure><h4>Conclusions</h4><p>Understanding a new application that you will develop and improve is not an easy task. In this article, we discuss how to comprehend and visualize the database model, but it is not exhaustive. On the database side, there are additional aspects that we haven’t covered here, such as <em>views</em>, <em>triggers</em>, and <em>procedures</em>. Additionally, in this example, we have declared all foreign keys at the database level, and this is not always the case in some applications.</p><p>If you’ve made it this far, thank you for your attention, and I hope this article has helped you understand your database model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52af01ac12fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tips for Using Django with PostgreSQL]]></title>
            <link>https://medium.com/joor-engineering/aaa-5600d82d9e63?source=rss-1a957b128e48------2</link>
            <guid isPermaLink="false">https://medium.com/p/5600d82d9e63</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[django]]></category>
            <dc:creator><![CDATA[David Garcia]]></dc:creator>
            <pubDate>Fri, 01 Dec 2023 09:04:00 GMT</pubDate>
            <atom:updated>2023-12-20T12:10:55.416Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>This article aims to demystify the process of creating models in Django, shedding light on crucial aspects that may not be immediately apparent. While it doesn’t replace the comprehensive Django ORM documentation, it serves as a valuable resource for understanding nuanced corner cases in model creation. Explore this guide for insights into optimizing your Django models and navigating scenarios that might not be covered in standard documentation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*olLEgbuAF9bQNyRJHRVsdA.png" /></figure><h3>PostgreSQL</h3><p>Let’s dive into PostgreSQL! This quick intro covers the basics without going too deep.</p><h4>Indexes</h4><p>While the benefits of establishing indexes in a database table are widely acknowledged, the drawbacks they may introduce are occasionally under-discussed:</p><ul><li><strong>Increased Storage:</strong> Indexes consume additional storage space. The more indexes you have, the more disk space your database will require. This can be a consideration if storage is a concern, especially in large databases.</li><li><strong>Overhead on Write Operations:</strong> When you insert, update, or delete records, the associated indexes need to be updated. This adds some overhead to write operations, potentially slowing them down.</li><li><strong>Maintenance Overhead:</strong> Indexes require maintenance during database updates. As data changes, the indexes must be kept in sync, and this process can introduce additional overhead.</li><li><strong>Choice of the Right Index:</strong> Selecting the appropriate type of index is crucial. Different types of indexes (e.g., B-tree, Hash, GiST, GIN) are suitable for different scenarios. The wrong choice might not provide the expected performance benefits.</li><li><strong>Index Fragmentation:</strong> Over time, indexes can become fragmented, impacting performance. Regular maintenance operations, such as VACUUM, help address this issue.</li></ul><blockquote>Indexes are good and necessary, but we must always keep in mind that they are not free.</blockquote><h4>Transactions</h4><p>While the intricacies of transactions, including their types and underlying mechanisms, can be extensively discussed, this section zooms in on data visibility across transactions. In PostgreSQL, initiating a transaction within another transaction introduces considerations of data visibility due to the concept of transaction isolation levels. PostgreSQL aligns with SQL standard principles, enforcing transaction and isolation standards prevalent in relational database systems.</p><p>Unlike some other database systems, PostgreSQL doesn’t support genuine <strong>nested transactions</strong>.</p><p>Transactions in PostgreSQL are essentially flat, lacking the ability to nest — a new transaction isn’t created within the scope of an existing one.</p><p>Nevertheless, PostgreSQL introduces a concept known as <a href="https://www.postgresql.org/docs/current/sql-savepoint.html"><strong>savepoints</strong></a>, enabling the creation of named points within a transaction for potential rollback. While akin to nested transactions, savepoints offer a means of partial rollback to a specific point within a transaction.</p><p>It’s crucial to understand that although savepoints facilitate a form of partial rollback, they don’t implement true nested transactions with distinct commit and rollback operations for each nesting level. Instead, each savepoint remains part of the overarching transaction, and any commit or rollback applies to the entire transaction.</p><h3>Django</h3><h4>Defining a model</h4><ul><li>Defines Django models using the models.Model class.</li><li>It uses specific fields from the Django model to represent data types, such as CharField, IntegerField, DateTimeField, etc.</li></ul><pre>from django.db import models<br><br>class MyDatabaseModel(models.Model):<br>    name = models.CharField(max_length=255)<br>    age = models.IntegerField()<br>    created_at = models.DateTimeField(auto_now_add=True)</pre><p>This model will generate the following table</p><pre>CREATE TABLE &quot;app_mydatabasemodel&quot; (<br>    &quot;id&quot; serial NOT NULL PRIMARY KEY,<br>    &quot;name&quot; varchar(255) NOT NULL,<br>    &quot;age&quot; integer NOT NULL,<br>    &quot;created_at&quot; timestamp with time zone NOT NULL DEFAULT NOW()<br>);</pre><p>If you don’t add a <strong>primary key</strong>, Django will add the id column with an autoincremental integer. In this case the id column</p><h4>Unique indexes</h4><p><strong>Single column</strong></p><ul><li>Use unique=True to create unique indexes on fields that must be unique in the table.</li></ul><pre>class MyModel(models.Model):<br>    name = models.CharField(max_length=255, unique=True)</pre><pre>CREATE TABLE &quot;app_mymodel&quot; (<br>    &quot;id&quot; serial NOT NULL PRIMARY KEY,<br>    &quot;name&quot; varchar(255) UNIQUE NOT NULL<br>);</pre><p>When a UNIQUE constraint is enforced, each new row insertion triggers a check for pre-existing values in the table. If the value already exists, the change is rejected, and an error is issued. This same validation process is applied when updating existing data.</p><p>Upon adding a UNIQUE constraint to a column or group of columns, PostgreSQL seamlessly generates a unique index for the specified column(s).</p><p>You never need to create an index on a UNIQUE field, since it already has its own <strong>index</strong>.</p><p><strong>Multiple columns</strong></p><ul><li>You can enforce uniqueness on multiple columns by using the unique_together option in the model&#39;s Meta class.</li></ul><pre>class MyModel(models.Model):<br>    name = models.CharField(max_length=255)<br>    category = models.CharField(max_length=255)<br><br>    class Meta:<br>        # Define unique constraint on &#39;name&#39; and &#39;category&#39;<br>        unique_together = [&#39;name&#39;, &#39;category&#39;]</pre><pre>CREATE TABLE &quot;myapp_mymodel&quot; (<br>    &quot;id&quot; serial NOT NULL PRIMARY KEY,<br>    &quot;name&quot; varchar(255) NOT NULL,<br>    &quot;category&quot; varchar(255) NOT NULL,<br>    CONSTRAINT &quot;myapp_mymodel_name_category_key&quot; UNIQUE (&quot;name&quot;, &quot;category&quot;)<br>);</pre><p>As in the preceding section, PostgreSQL will automatically generate an index for these two fields. The indexing sequence will prioritise the name field followed by the category field.</p><p>When dealing with a <strong>composite index </strong>(an index spanning multiple columns) and one of those columns is already encompassed by a <strong>single-column index</strong>, establishing a separate index on that individual column may prove <strong>redundant</strong>. This redundancy arises from the fact that the composite index inherently incorporates the coverage of the indexed column.</p><p>In the given example, the index created by the constraint already encompasses queries solely focused on the <em>name</em>. Consequently, creating an additional index specifically for the <em>name</em> is <strong>unnecessary</strong>.</p><h4>Relationships between models</h4><ul><li>Use relationship fields such as ForeignKey or ManyToManyField to establish relationships between models.</li><li>Django automatically creates foreign keys and intermediate relationship tables as needed.</li></ul><pre>class Author(models.Model):<br>    name = models.CharField(max_length=255)<br><br>class Book(models.Model):<br>    title = models.CharField(max_length=255)<br>    author = models.ForeignKey(Author, on_delete=models.CASCADE)</pre><pre>CREATE TABLE &quot;myapp_author&quot; (<br>    &quot;id&quot; serial NOT NULL PRIMARY KEY,<br>    &quot;name&quot; varchar(255) NOT NULL<br>);<br><br>CREATE TABLE &quot;myapp_book&quot; (<br>    &quot;id&quot; serial NOT NULL PRIMARY KEY,<br>    &quot;title&quot; varchar(255) NOT NULL,<br>    &quot;author_id&quot; integer NOT NULL REFERENCES &quot;myapp_author&quot; (&quot;id&quot;) DEFERRABLE INITIALLY DEFERRED<br>);<br><br>CREATE INDEX &quot;app_libro_autor_id&quot; ON &quot;myapp_book&quot; (&quot;author_id&quot;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/461/1*DY-ErjnqKqn2PljVO5jq9A.png" /></figure><p>The field that you define as a foreign key, Django will add to the database as a suffix _id. In this example Book.author is author_id in the database.</p><p>By default, when you define a foreign Django will create an index. If you don’t want create this index you can pass the parameter db_index=False to models.ForeignKey(...) function.</p><p>Let’s revisit the ‘<em>Unique Indexes</em>’ section. If we require a <strong>foreign key</strong> to be <strong>unique</strong> in a table, we need to instruct Django not to create an index on this foreign key (<em>db_index=False</em>), as PostgreSQL automatically creates one by default.</p><p>For instance, if we aim to prevent publishing more than one book with the same name per author, we would create an <em>UNIQUE</em> index using the tuple of <em>author</em> and <em>title</em>. In such cases, the constraint index generated by PostgreSQL will encompass the foreign key index generated by Django. Therefore, it’s necessary to inform Django not to create this index.</p><p>Similarly, when dealing with a <strong>composite index</strong> (an index on multiple columns) where the first part is a <strong>foreign key</strong>, Django should be directed not to create an index on this foreign key. This is because the composite index will inherently cover the foreign key index.</p><p>Consider the scenario where we want efficient searches for books by author_id and title. Here, creating an index with the tuple <em>author_id</em> and <em>title</em> will suffice, and the composite index will encompass the foreign key index created by Django. Therefore, it’s important to instruct Django not to create this additional index.</p><h4>Database architectures</h4><p><strong>Main-Replica</strong></p><p>Commonly, your production database was designed in a main-replica architecture (write in the main node, read from the replica nodes). The frameworks, like Django, allow you to configure the database connection based on this architecture. This architecture works very well but there is a common problem that few developers think when they are developing an application: Data Consistency.</p><p>In the case of the Django app, sometimes it is very common that in the same request, you write and read. Due to the inherent asynchrony between the main (where writes occur) and replica (where reads occur), there may be latency in data replication. If you read immediately after writing in the same request, you might obtain outdated data on the replicas before the writes are replicated.</p><p>In scenarios where you need to perform both database writes and reads within the same request, please ensure that both operations are directed toward the same database node.</p><p>We need to consider this factor because, at times, it can lead to reporting <strong>peculiar bugs</strong> that are challenging to debug.</p><h3>Recommended Lectures &amp; Courses</h3><ul><li><a href="https://www.oreilly.com/library/view/designing-data-intensive-applications/9781491903063/">Designing Data-Intensive Applications</a></li><li><a href="https://www.udemy.com/course/database-engines-crash-course/">Fundamentals of Database Engineering</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5600d82d9e63" width="1" height="1" alt=""><hr><p><a href="https://medium.com/joor-engineering/aaa-5600d82d9e63">Tips for Using Django with PostgreSQL</a> was originally published in <a href="https://medium.com/joor-engineering">JOOR Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>