<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Christopher Gathuthi on Medium]]></title>
        <description><![CDATA[Stories by Christopher Gathuthi on Medium]]></description>
        <link>https://medium.com/@soverignchriss?source=rss-318880a6cb12------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*WialPzvBO5mOGAwg</url>
            <title>Stories by Christopher Gathuthi on Medium</title>
            <link>https://medium.com/@soverignchriss?source=rss-318880a6cb12------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:07:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@soverignchriss/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[TCP 3-way handshake process]]></title>
            <link>https://medium.com/@soverignchriss/tcp-3-way-handshake-process-f9e78e6133e4?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9e78e6133e4</guid>
            <category><![CDATA[tcp]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Sat, 01 Jun 2024 08:17:32 GMT</pubDate>
            <atom:updated>2024-06-01T08:17:32.099Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QydRbrce4p0fJ-5ZB_UTTw.png" /><figcaption>TCP 3-way handshake process</figcaption></figure><p><strong><em>What is TCP</em></strong></p><p>Transmission Control Protocol (TCP) is a layer 4 protocol of the Open System Interconnection (OSI) model. TCP is the most reliable protocol available and many applications are built on top of TCP protocol. TCP is a stateful protocol, meaning the client and the server store information about each other. The TCP handshake process creates the state between the client and the server.</p><p><strong><em>TCP handshake process</em></strong></p><p>The TCP handshake process begins on the client side. The client initiates a connection to the server by sending the Initial Sequence Number to the server (ISN). This process of sending the ISN to the server is called SYN. The ISN is a positive integer generated by the client&#39;s operating system. The server will receive the ISN and add one to that value. For instance, if the ISN number is 10 the server will add one totaling 11. The one integer, added to the ISN is called a ghost byte. Once the server has received the SYN from the client, it will send two items back to the client; one is the acknowledgment for the SYN received and the server ISN number hence this process is called a SYN-ACK. The acknowledgment number is the number ISN plus the added ghost byte and the SYN is the server’s ISN value. When the client received both SYN and ACK values. It will send an ACK to the server to acknowledge that it has received the ISN number from the server.</p><p>Each time a client makes a TCP connection if the TCP segment has any data, the server will increment the value based on the size of the packet. For instance, if the ISN number is 200 and the size of the TCP segment is 30bytes. The server will increment the ISN number by 30 in turn the ACK value will be 230. You may ask yourself what if the TCP is not carrying any data? Which value will the server use to increment the value of the client ISN? The server by default always increments by one byte. This byte is called a ghost byte and this happens only once when the client first initiates the TCP handshake.</p><p><strong><em>SYN flood attack (half-open attack)</em></strong></p><p>SYN flood attack is a direct denial of service attack that aims to make the server unavailable to legitimate traffic by consuming all available server resources. When a client initiates a SYN process with the server, the kernel of the server will create a SYN queue, and the SYN values are added to the SYN queue. The server will send a SYN-ACK to the client and the client will send an ACK back to the server and the kernel of the server operating system will remove the client’s SYN ISN from the SYN queue and put it in the accept queue thereby freeing the SYN queue for more SYN connection. For SYN flooding to occur, the client fails to send an ACK to the server. This failure makes the SYN queue full because the kernel cannot remove the SYN in the queue if the client has not acknowledged it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9e78e6133e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real-time Notification Streaming using SSE and Htmx]]></title>
            <link>https://medium.com/@soverignchriss/real-time-notification-streaming-using-sse-and-htmx-32798b5b2247?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/32798b5b2247</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[htmx]]></category>
            <category><![CDATA[server-side-rendering]]></category>
            <category><![CDATA[django]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Sun, 14 Jan 2024 21:00:07 GMT</pubDate>
            <atom:updated>2024-01-14T21:00:07.690Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/972/1*k6Yv_I28SUhSu6tMfZcCpQ.png" /><figcaption>banner</figcaption></figure><p>In this article, I will demonstrate how to stream notifications using server-sent events and using htmx to render the data being transmitted to the client.</p><p>Server-sent events use HTTP protocol to communicate with the client. In a normal request-response cycle. A client being a browser will make a request to the server and the server will respond to the request with a response and the cycle is complete and comes to an end. In server-sent events, the client will make a request to the server, and get the response but the response will never end it will be a continuous stream of response unless the client and server connection is terminated. It is a long unending response from the server. This unending response contains a content-type of <strong>text/event-stream.</strong></p><p>Htmx will be responsible for parsing those responses and injecting the response into the DOM. A typical response for a server-sent event contains the following.</p><pre>data: &lt;html elements&gt; event: &lt;eventName&gt;</pre><p>Each response must have data and the event.</p><ol><li><strong>project setup</strong></li></ol><p>In the article, I will use the Django web framework for developing the backend for streaming events and the htmx library for parsing the events and injecting HTML elements in the DOM. In htmx 2.0, server-sent events are migrated to extensions. If you are loading htmx via CDN remember to add the htmx extensions CDN.</p><p>Our application will be a simple blog application that will have the following features.</p><p>a) Reacting to a blog post (like/disliking)</p><p>b) Streaming notification to the signed-in user, the user will be notified when someone reacts to their posts.</p><p><strong>2. Defining the models</strong></p><p>In Django, models are used to define the database tables. The following will be our models.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5fc82b3f677dfa43201a07f1620c0bb6/href">https://medium.com/media/5fc82b3f677dfa43201a07f1620c0bb6/href</a></iframe><p><strong>3. Defining the views for handling requests and responding to requests.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a74d6942adb11e5d5390448df749bde8/href">https://medium.com/media/a74d6942adb11e5d5390448df749bde8/href</a></iframe><p>The view file has two view classes. The <strong>BlogView </strong>and <strong>NotificationView </strong>which are responsible for rendering Blog pages and generating server-sent events respectively.</p><p><strong><em>BlogView</em></strong></p><p>The BlogView class is responsible for rendering blog posts to the client. The post method will accept the reaction(sentiment) from the user. When the user likes or dislikes a post. A post request is sent to the backend which has the ID of a blog and the sentiment of that blog. Using the ID passed in the request, the blog that the user reacted to is retrieved and the blog instance is used to establish the relationship. Using the sentiment, the retrieved blog instance, and the user instance from the request a Reaction object is created. Depending on the type of reaction a html response is sent to the client.</p><p><strong><em>NotificationView</em></strong></p><p>This class view is responsible for creating the server-sent event. In the get method, the respone object is an instance of<strong> StreamingHttpRespone </strong>it takes a generator as an argument which is called and it is responsible for rendering the response content. From the code snippet above, StreamingHttpResponse takes a function called <strong>event_stream(). </strong>This function takes the request object to retrieve the user instance from the request body and iterates through all the user’s Notification that has the <strong>is_read=False</strong> attribute. When we filter out all the notifications we will loop through the query set and render each notification object to the user. If the query set is empty then we will render an empty string.</p><p>The post method in the NotificationView class is responsible for changing the status of the Notification from <strong>is_read = False </strong>to <strong>is_read = True. </strong>The reason for updating it is to prevent the user from receiving the same notification twice. The post method returns an empty response. The method <strong>render_to_string </strong>returns a HTML file which is sent to the client browser as a notification pop-up.</p><pre>yield f&quot;data: {html} \n\n&quot; </pre><p>The above line denotes the end of a single stream a stream is separated with a double newline. Take note of empty lines in the HTML file returned by the <strong>render_to_string</strong> method, and make sure there is no empty line separating HTML elements.</p><p><strong>3. Notification signal.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/91752862e747200fb0cbe63e479fa7d0/href">https://medium.com/media/91752862e747200fb0cbe63e479fa7d0/href</a></iframe><p>The <strong>create_notification </strong>listens to the <strong>post_save</strong> signal from the Reaction model. When a reaction object is created after a user has reacted to a blog post, a post_save signal is dispatched then a notification instance is created.</p><p><strong>4. Templates</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5abf698716d6088bef7f973f05356313/href">https://medium.com/media/5abf698716d6088bef7f973f05356313/href</a></iframe><p>This is the home page that is responsible for rendering the blog posts and the notification to the user. I will narrow down and focus on two HTML segments that demonstrate the use of htmx.</p><p><strong>a) htmx with sse</strong></p><pre>&lt;div class=&quot;w-1/4 float-right&quot; hx-ext=&quot;sse&quot; sse-connect=&quot;{% url &#39;notification&#39; %}&quot; hx-swap=&quot;afterbegin&quot;&gt;<br>    &lt;div sse-swap=&quot;message&quot;&gt;<br><br>    &lt;/div&gt;<br><br>&lt;/div&gt;</pre><p>The parent element has some htmx attributes that enable the client to receive the server-sent events.</p><ol><li><strong>hx-ext=”sse” </strong>This attribute will install the htmx extension to the HTML element.</li><li><strong>sse-connect=”{% url ‘notification’ %}” </strong>This defines the URL for communicating with the backend.</li><li><strong>hx-swap=”afterbegin” </strong>This attribute defines how the response from the backend will be injected into the DOM. The <strong>afterbegin </strong>behaviour will <br>prepend the content of the response before the first child inside the target therefore the latest notification will be a the top of the previous notification hence a chronological order of notification.</li><li><strong>sse-swap=”message” </strong>This attribute takes an eventName, by default, it takes an event name <strong>message </strong>if the response has no defined event name. The event name is sent together with the response data by the SSE response. In our case, we did not define the event name therefore we are using the default event name set by the browser. In all responses, htmx will parse the event name and then look for the element that has the attribute <strong>sse-swap=”eventName” </strong>whose sse-swap value is similar to the event name sent by the SSE response and then swap it. This is akin to <strong>hx-swap-oob=”true”</strong>. This attribute must be placed <strong>on</strong> or <strong>inside </strong>the tag containing the hx-ext attribute.</li></ol><p><strong>b) htmx sending user reactions</strong></p><pre>&lt;button class=&quot;h-10&quot; hx-post=&quot;{% url &#39;home&#39; %}&quot;  hx-vals=&#39;{&quot;id&quot;:&quot;{{blog.id}}&quot;, &quot;sentiment&quot;:&quot;like&quot;}&#39;&gt;&lt;i class=&quot;fa-solid fa-thumbs-up text-3xl&quot;&gt;&lt;/i&gt;&lt;/button&gt;<br>&lt;button class=&quot;h-10&quot; hx-post=&quot;{% url &#39;home&#39; %}&quot;  hx-vals=&#39;{&quot;id&quot;:&quot;{{blog.id}}&quot;, &quot;sentiment&quot;:&quot;dislike&quot;}&#39;&gt;&lt;i class=&quot;fa-solid fa-thumbs-down text-3xl&quot;&gt;&lt;/i&gt;&lt;/button&gt;  </pre><p>The above snippet is of HTML button tags for liking and disliking respectively. You may wonder how does a button tag sends a post request to the backend yet we are not using any form element as usual. To answer this, this is why htmx exists. Htmx enables HTML elements to send requests to the backend using Ajax. Any element can send a request provided it has htmx attributes. To make a post request to Django, one must include the CSRF token in the request header and htmx has <strong>hx-headers</strong> attribute used for adding headers to<strong> the hx-post </strong>request. If you look at the code above you will note that we have not added the <strong>hx-headers </strong>attribute because we have defined it at the top-level parent element which is the body tag. The<strong> hx-headers</strong> attribute will be inherited by all child elements therefore to avoid duplicating the code we declare it at the body tag.</p><pre>&lt;body hx-headers=&#39;{&quot;X-CSRFToken&quot;: &quot;{{ csrf_token }}&quot;}&#39;&gt;<br>{% block main %}<br>{% endblock main %}<br>&lt;/body&gt;</pre><p><strong>Notification template</strong></p><pre>&lt;div id=&quot;toast-notification&quot; hx-post=&quot;{% url &#39;read-notification&#39; %}&quot; hx-trigger=&quot;load once&quot;<br>    hx-vals=&#39;{&quot;id&quot;:&quot;{{notification.id}}&quot;}&#39; hx-swap=&quot;none&quot;<br>    class=&quot;w-full max-w-xs p-4 text-gray-900 bg-white rounded-lg shadow dark:bg-gray-800 dark:text-gray-300&quot;<br>    role=&quot;alert&quot;&gt;<br>    &lt;div class=&quot;flex items-center mb-3&quot;&gt;&lt;span class=&quot;mb-1 text-sm font-semibold text-gray-900 dark:text-white&quot;&gt;New<br>            notification&lt;/span&gt;&lt;button type=&quot;button&quot;<br>            class=&quot;ms-auto -mx-1.5 -my-1.5 bg-white justify-center items-center flex-shrink-0 text-gray-400 hover:text-gray-900 rounded-lg focus:ring-2 focus:ring-gray-300 p-1.5 hover:bg-gray-100 inline-flex h-8 w-8 dark:text-gray-500 dark:hover:text-white dark:bg-gray-800 dark:hover:bg-gray-700&quot;<br>            data-dismiss-target=&quot;#toast-notification&quot; aria-label=&quot;Close&quot;&gt;&lt;span class=&quot;sr-only&quot;&gt;Close&lt;/span&gt;&lt;svg<br>                class=&quot;w-3 h-3&quot; aria-hidden=&quot;true&quot; xmlns=&quot;http://www.w3.org/2000/svg&quot; fill=&quot;none&quot; viewBox=&quot;0 0 14 14&quot;&gt;<br>                &lt;path stroke=&quot;currentColor&quot; stroke-linecap=&quot;round&quot; stroke-linejoin=&quot;round&quot; stroke-width=&quot;2&quot;<br>                    d=&quot;m1 1 6 6m0 0 6 6M7 7l6-6M7 7l-6 6&quot; /&gt;<br>            &lt;/svg&gt;&lt;/button&gt;&lt;/div&gt;<br>    &lt;div class=&quot;flex items-center&quot;&gt;<br>        &lt;div class=&quot;relative inline-block shrink-0&quot;&gt;&lt;img class=&quot;w-12 h-12 rounded-full&quot;<br>                src=&quot;https://randomuser.me/api/portraits/men/81.jpg&quot; alt=&quot;Jese Leos image&quot; /&gt;&lt;span<br>                class=&quot;absolute bottom-0 right-0 inline-flex items-center justify-center w-6 h-6 bg-blue-600 rounded-full&quot;&gt;&lt;svg<br>                    class=&quot;w-3 h-3 text-white&quot; aria-hidden=&quot;true&quot; xmlns=&quot;http://www.w3.org/2000/svg&quot; viewBox=&quot;0 0 20 18&quot;<br>                    fill=&quot;currentColor&quot;&gt;<br>                    &lt;path<br>                        d=&quot;M18 4H16V9C16 10.0609 15.5786 11.0783 14.8284 11.8284C14.0783 12.5786 13.0609 13 12 13H9L6.846 14.615C7.17993 14.8628 7.58418 14.9977 8 15H11.667L15.4 17.8C15.5731 17.9298 15.7836 18 16 18C16.2652 18 16.5196 17.8946 16.7071 17.7071C16.8946 17.5196 17 17.2652 17 17V15H18C18.5304 15 19.0391 14.7893 19.4142 14.4142C19.7893 14.0391 20 13.5304 20 13V6C20 5.46957 19.7893 4.96086 19.4142 4.58579C19.0391 4.21071 18.5304 4 18 4Z&quot;<br>                        fill=&quot;currentColor&quot; /&gt;<br>                    &lt;path<br>                        d=&quot;M12 0H2C1.46957 0 0.960859 0.210714 0.585786 0.585786C0.210714 0.960859 0 1.46957 0 2V9C0 9.53043 0.210714 10.0391 0.585786 10.4142C0.960859 10.7893 1.46957 11 2 11H3V13C3 13.1857 3.05171 13.3678 3.14935 13.5257C3.24698 13.6837 3.38668 13.8114 3.55279 13.8944C3.71889 13.9775 3.90484 14.0126 4.08981 13.996C4.27477 13.9793 4.45143 13.9114 4.6 13.8L8.333 11H12C12.5304 11 13.0391 10.7893 13.4142 10.4142C13.7893 10.0391 14 9.53043 14 9V2C14 1.46957 13.7893 0.960859 13.4142 0.585786C13.0391 0.210714 12.5304 0 12 0Z&quot;<br>                        fill=&quot;currentColor&quot; /&gt;<br>                &lt;/svg&gt;&lt;span class=&quot;sr-only&quot;&gt;Message icon&lt;/span&gt;&lt;/span&gt;&lt;/div&gt;<br>        &lt;div class=&quot;ms-3 text-sm font-normal&quot;&gt;<br>            &lt;div class=&quot;text-sm font-semibold text-gray-900 dark:text-white&quot;&gt;Bonnie Green&lt;/div&gt;<br>            &lt;div class=&quot;text-sm font-normal&quot;&gt;{{notification.message}}&lt;/div&gt; &lt;span<br>                class=&quot;text-xs font-medium text-blue-600 dark:text-blue-500&quot;&gt;{{ notification.timestamp|timesince}}<br>                ago&lt;/span&gt;<br>        &lt;/div&gt;<br>    &lt;/div&gt;<br>&lt;/div&gt;</pre><p>The HTML snippet above is a notification fragment. This notification will be sent to the user when a notification instance exists.</p><pre>&lt;div id=&quot;toast-notification&quot; hx-post=&quot;{% url &#39;read-notification&#39; %}&quot; hx-trigger=&quot;load once&quot;<br>    hx-vals=&#39;{&quot;id&quot;:&quot;{{notification.id}}&quot;}&#39; hx-swap=&quot;none&quot;<br><br>    ....<br>&lt;/div&gt;</pre><p>The code above is for the notification parent tag. It has the following htmx attributes.</p><p><strong>a) hx-post=”{% url ‘read-notification’ %}</strong></p><p><strong>b) hx-trigger=”load once” </strong>This attribute is responsible for sending a request to change the <strong>is_read=False</strong> to <strong>is_read=True</strong>. The load once value means, that when the notification pop-up is loaded into the DOM, it will initiate a one-time request to the backend to change the notification attribute so that the notification won’t be rendered twice because we are filtering out notifications that have <strong>is_read=False</strong>, When the notification is sent to the user browser, the user is responsible for dismissing the notification.</p><p>To avoid a scenario where multiple requests are sent to the backend by the pop-up since the pop-up will initiate requests once it is loaded into the DOM. We include the <strong>once </strong>modifier hence <strong>“load once” </strong>therefore only one request will be sent<strong>.</strong></p><p>c)<strong> hx-swap=”none”</strong> This attribute specifies how the response will be injected into the DOM. The value none prevents htmx from injecting the response to the DOM. In our case, we are returning an empty string once we change the notification <strong>is_read</strong> attribute. We don’t need the response and since the browser uses the request-response cycle, we cannot prevent the server from sending us a response but we can prevent the browser from using the response hence <strong>none</strong> value will not inject the response.</p><p>d) <strong>hx-vals=’{“id”:”{{notification.id}}”}’ </strong>This attribute is used to send additional data together with the request. Remember we are not using a form element to send data to the backend and htmx provides a way of adding parameters that will be submitted with each request. <strong>hx-vals</strong> takes JSON formatted values. The values can be fetched from the POST query dictionary or the <strong>request.body. </strong>The submitted data will contain the notification ID which will be used to retrieve the notification instance and change the value of <strong>is_read</strong> attribute.</p><p><strong>Conclusion</strong></p><p>Server-sent event is an old technology that is still useful in developing web applications. Its application is not limited to notifications only, in instances where lazy loading is required for instance sending large responses you can use SSE. As much as SSE is used for real-time communication, a client has to be online all the time meaning the client and server must be connected therefore when designing your application put htmx SSE at the HTML element that is inherited to all pages in the application so that the connection to the server is uninterrupted and available. For instance, you can put it in the header part of your application.</p><p><strong>Further reading</strong></p><ul><li><a href="https://htmx.org/extensions/server-sent-events/">The server-sent-events Extension</a></li><li><a href="https://docs.djangoproject.com/en/5.0/">Django</a></li><li><a href="https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events">Using server-sent events - Web APIs | MDN</a></li><li><a href="https://github.com/chrisgathuthi/htmxsse.git">GitHub - chrisgathuthi/htmxsse</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32798b5b2247" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Chat Application with Django-channels & Htmx]]></title>
            <link>https://medium.com/@soverignchriss/chat-application-with-django-channels-htmx-dd8174f59a03?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd8174f59a03</guid>
            <category><![CDATA[websocket]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[django-htmx]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Thu, 30 Nov 2023 16:01:54 GMT</pubDate>
            <atom:updated>2023-11-30T16:01:54.007Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ruxX_5TyIWPDN0mx8VbfUg.png" /><figcaption>WebSockets icon(left) &amp; HTMX icon (right)</figcaption></figure><p>WebSocket is a computer communications protocol, that provides simultaneous two-way communication channels over a single Transmission Control Protocol connection. WebSockets enable communication between the server and the client without having to poll the server for the response.</p><p>In Django, Channels wraps Django’s native asynchronous view support, allowing Django projects to handle not only HTTP, but protocols that require long-running connections too — WebSockets, MQTT, chatbots, amateur radio, and more. WebSockets are popularly used when there’s a need to have real-time communication such as chatting applications or streaming events from the server. In this article, I am using WebSockets to demonstrate its use in developing a chat application.</p><p>We will start by installing django and django-channels. We will use those libraries to create the application. I am assuming that you are familiar with setting up a Django project and creating a Python environment.</p><pre>#installing dependancies<br>python -m pip install django <br>python -m pip install -U &#39;channels[daphne]&#39;</pre><p>Notice ‘<strong>channels[daphne]’</strong> this will install channels as well as the <strong>daphne</strong> ASGI server for development purposes. Daphne server will take over the management command from Django’s default management command. The official channels library uses the <strong>daphne</strong> development server but you can also install the ASGI server you want. Daphne server is placed at the top in the installed app list in Django settings.</p><p>After installation, we will add channels to the installed apps in Django settings.</p><pre># Application definition<br><br>INSTALLED_APPS = [<br>    &#39;daphne&#39;,<br>    &#39;django.contrib.admin&#39;,<br>        ...<br>    &#39;chat.apps.ChatConfig&#39;, #chat application<br>     <br>]</pre><p>In the Django <strong>myproject</strong>/<strong>asgi.py </strong>file we will add the following lines of code.</p><pre>import os<br><br>from channels.routing import ProtocolTypeRouter<br>from django.core.asgi import get_asgi_application<br><br>os.environ.setdefault(&#39;DJANGO_SETTINGS_MODULE&#39;, &#39;mysite.settings&#39;)#replace with your app settings<br># Initialize Django ASGI application early to ensure the AppRegistry<br># is populated before importing code that may import ORM models.<br>django_asgi_app = get_asgi_application()<br><br>application = ProtocolTypeRouter({<br>    &quot;http&quot;: django_asgi_app,<br>    # Just HTTP for now. (We can add other protocols later.)<br>})</pre><p>We will update the <strong>settings.py </strong>file and add the following configuration to point to the ASGI application that we have created above so that all aync requests in Django in our case, it is the WebSockets will be routed to the ASGI application.</p><pre>ASGI_APPLICATION = &quot;myproject.asgi.application&quot;</pre><p>After we have set up channels and Django we have to define the purpose of our application. It will be a simple chat application that will have two models; Message and room models. We will also create a template that will have the chat page. In the chat page, we will add a link for htmx cdn.</p><p>Let&#39;s define the models.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce232eedbecacb278185c0f887b7cf8b/href">https://medium.com/media/ce232eedbecacb278185c0f887b7cf8b/href</a></iframe><p>In the file above we have three models; one is the user model we have imported above and the second one is the Room model, the room model will have the chat groups think of it as the usual chat groups found in popular social media such as WhatsApp and Telegram and lastly we have the Message model which will contain the content of the chat that is the message being passed from one User to another.</p><p>What we are trying to achieve is a user will log in to the application proceed to join a Room which will be a chat group then proceed to write a message in the chat and it will be broadcast to all the users in the group.</p><p>Let’s create a template for the chatting page. To create a template we have to create its view function that will render the page. We will use Tailwind CSS to style our page.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/02c7738d76f274c64eee2f7798afe98b/href">https://medium.com/media/02c7738d76f274c64eee2f7798afe98b/href</a></iframe><p>Notice in the base.html file I have included the htmx cdn.</p><pre>@login_decorator<br>def talk(request):<br>    &quot;&quot;&quot;chat page view&quot;&quot;&quot;<br>    return render(request, &quot;.html&quot;, {&#39;groups&#39;:Room.objects.all()}) </pre><p>The method above will render the <strong>chat.html </strong>page. We are rendering all the Room objects that we have in our database. For the purpose of simplicity, we are going to create the room objects using the Django admin page. Create several rooms such as music, movies, or games. Remember those rooms will be the chat groups. Remember to add the <strong>@login_decorator</strong> to limit only authenticated users only. Therefore we will have to create the authentication system.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1a4835bfcd576964ca5e377ec98f783/href">https://medium.com/media/b1a4835bfcd576964ca5e377ec98f783/href</a></iframe><p>The above code demonstrates a simple authentication system for our application. Users must be authenticated to use the app therefore we will use the login required decorator.</p><p>Let’s focus on channels. To use channels we have to create two files <strong>routing.py </strong>and <strong>consumers.py </strong>those two files are akin to Django <strong>urls.py </strong>and <strong>views.py </strong>respectively. Let us define the consumers.py file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0efcec40f825bb0c4e2bd03fbd74efa/href">https://medium.com/media/e0efcec40f825bb0c4e2bd03fbd74efa/href</a></iframe><p>In the consumers.py file, we are creating a Class called <strong>TalkConsumer</strong> that inherits from the <strong>WebSocketConsumer</strong> class. The Talk class will be responsible for receiving and sending chats using the WebSockets.</p><p>TalkConsumer class has only three main methods.</p><ol><li><strong>connect(self)</strong></li></ol><p>This method is first called when the client is initially connected such as accepting or rejecting the connection based on certain criteria or sending an initial message to the client. From our example above this method does the following. It gets the room ID of a room and then creates a group name. A room instance is retrieved from the <strong>scope</strong>. A scope has all the information about a WebSocket connection. It is similar to requests in Django views. From the scope, we also retrieve the user instance that is logged in. <strong>self.accept(), </strong>this method accepts a WebSocket connection therefore initiating a bi-directional connection with the client. After the connection is created a user is added to a group and remember the group is the rooms that we retrieved based on the ID sent by the WebSocket and in channels they are called a group. Channels gives an arbitrary name to the group but we want to control the naming of our group that’s why we are using rooms and the group names.</p><pre> # join the room/group<br>  async_to_sync(self.channel_layer.group_add)(self.group_name, self.channel_name)</pre><p><strong>aysnc_to_sync </strong>is a method imported from the <strong>async.grief </strong>in Django. It converts an async method to a synchronous method. It converts <strong>channel_layer.group_add</strong> to synchronous and then <strong>(self.group_name, self.channel_name) </strong>takes the group name and the channel_name from the scope and adds the channel name to the group. One thing to note is that each connection is established via a WebSocket a unique channel is created and now we are putting all those unique channels into one group which will enable all those channels to talk to other channels (users) who have their own channels and by doing that. That’s how a chatting feature is created.</p><p>You may ask, why do we have to convert the asynchronous method to the synchronous method? Because channels and Django are different, that is Django executes code synchronously for instance when accessing the database but channels will use a channel layer which channels will have to access asynchronously.</p><p>Before we proceed further, we have to configure a channel layer. A channel layer is a kind of communication system, which allows multiple parts of our application to exchange messages, without shuttling all the messages or events through the database. We need a channel layer to give consumers the ability to talk to one another.</p><pre>docker run --rm -p 6379:6379 redis:7<br><br>python3 -m pip install channels_redis</pre><p>From the snippet above we have created a docker container that has a Redis image which will be used as a channels layer and then we have installed the <strong>channels_redis </strong>which will interface our Django application with Redis. After installing Redis and channels_redis we have to configure it in our settings file.</p><pre>CHANNEL_LAYERS = {<br>    &quot;default&quot;: {<br>        &quot;BACKEND&quot;: &quot;channels_redis.core.RedisChannelLayer&quot;,<br>        &quot;CONFIG&quot;: {<br>            &quot;hosts&quot;: [(&quot;127.0.0.1&quot;, 6379)],<br>        },<br>    },<br>}</pre><pre>html = get_template(&quot;partial/join.html&quot;).render(context={&quot;user&quot;:self.user})<br>self.send(text_data=html)</pre><p>From the snippet above we are using a template file called <strong>join.html </strong>passing the user instance from the scope and sending it to the client side this template is a notification to all group members and it will notify the name of the user who has joined the room.</p><p>2. d<strong>isconnect(self)</strong></p><p>This method is called when the client terminates the connection, it takes the close_code parameter which is an integer that denotes the reason why the connection is closed. This method has the following function inside the function block.</p><pre>async_to_sync(self.channel_layer.group_discard)(self.group_name,self.channel_name)<br>html = get_template(&quot;partial/leave.html&quot;).render(context={&quot;user&quot;:self.user})<br>        self.send(<br>            text_data=html<br>        )<br> self.room.online.remove(self.user)</pre><p>The method above will remove the user from the group and send a notification to the Group on the client side with a message the user has left the group and the user is removed from the room.</p><p>3. <strong>receive(self, text_data=None, bytes_data=None)</strong></p><p>This method is invoked when the consumer receives a message from the client over the WebSocket. It handles the incoming data and defines how the server responds to these messages. In our case, the method is deserializing the client&#39;s message and creates an instance of the message then it responds with a html file that has all the messages of that particular room. It uses a reverse relationship to get all messages for a particular room.</p><p>Our focus here will be on htmx. Htmx is a library that allows you to access modern browser features directly from HTML, rather than using JavaScript.</p><p>On our chat page, there’s a code block that entails all groups available in our application.</p><pre> {% for group in groups %}<br>  &lt;button class=&quot;flex flex-row items-center hover:bg-gray-100 rounded-xl p-2&quot;<br>         hx-post=&quot;{% url &#39;group&#39; group.id %}&quot; hx-target=&quot;#chat-window&quot;<br>         hx-headers=&#39;{&quot;X-CSRFToken&quot;: &quot;{{ csrf_token }}&quot;}&#39;&gt;<br>      &lt;div class=&quot;flex items-center justify-center h-8 w-8 bg-gray-200 rounded-full&quot;&gt;<br>                                {{forloop.counter}}<br>          &lt;/div&gt;<br>      &lt;div class=&quot;ml-2 text-sm font-semibold&quot;&gt;{{group.name}}&lt;/div&gt;<br>         &lt;div class=&quot;flex items-center justify-center ml-auto text-xs text-white bg-red-500 h-4 w-4 rounded leading-none&quot;&gt;<br>                                {{group.get_online_count}}<br>            &lt;/div&gt;<br>   &lt;/button&gt;<br>{% endfor %}</pre><p>This section has some htmx attributes that are similar to HTML tag attributes and they are used as HTML tag attributes.</p><ul><li><strong>hx-post = “{% url ‘group’ group.id %}” — </strong>send a post request to the group URL with a parameter of group id.</li><li><strong>hx-target= “#chat-window” — </strong>swap the response to a div with the id of <strong>“#chat-window”.</strong></li><li><strong>hx-headers=”{ ‘X-CSRFToken”: ’{{csrf_token}}’ }” — </strong>this attribute is used to add a CSRF token to the request header. We are adding a CSRF token because this is a POST request.</li></ul><p>The response from that POST request will be swapped in the inner part of <strong>#chat-window </strong>element.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9RctTczBjtbgSs4itnKxpA.png" /><figcaption>chat page</figcaption></figure><p>On the bottom left are the various rooms/groups called active communities that I have created. The response from the server that will sent once we click on any group, will be swapped on the right side of the page where we have an empty grey space. For instance, I have clicked on dev group and the response will look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I75AGqTvo2dS0-fj_n6Z_g.png" /></figure><p>Notice on the top right is a notification that a user named Chris has joined the group, That is a notification sent when the user first joins the group and all users present in the group are able to see it. and at the bottom part, we have the chat input element tag where a message will be written and sent and the response will appear on the grey shaded area.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZqGu2pM_fEEZnI7CIcsX8Q.png" /></figure><p>The response that was sent by the server when we clicked on the dev group it contains some htmx attributes that enable us to use WebSockets and establish a connection to the server.</p><pre>&lt;div class=&quot;flex flex-col flex-auto flex-shrink-0 rounded-2xl bg-gray-100 h-full p-4&quot; hx-ext=&quot;ws&quot;<br>ws-connect=&quot;/talk/{{room_name}}/&quot;&gt;</pre><p>The <strong>ws-connect=’/talk/{{room_name}}/” </strong>will establish a connection to the WebSocket’s URL. Remember, we already defined the WebSocket’s URL in the routing.py file. The <strong>{{room_name}} </strong>is the room ID or the group ID of the room we joined. The <strong>{{room_name}} </strong>context variable is sent by the view that sends a response when the group is selected.</p><pre>@login_required<br>def group(request, group_name):<br>    &quot;&quot;&quot;chat window view&quot;&quot;&quot;<br><br>    group = Room.objects.get(id=group_name)<br>    messages = group.message_set.all()<br>    return render(request, &quot;partial/group-chat.html&quot;,{&quot;messages&quot;:messages, &quot;room_name&quot;:group_name,&quot;name&quot;:group})</pre><p>From the server response above we have the following snippet of code in the form element.</p><pre>&lt;form id=&quot;chat-form&quot; class=&quot;w-100&quot;&gt;<br>                &lt;input type=&quot;text&quot;<br>                    class=&quot;flex w-full border rounded-xl focus:outline-none focus:border-indigo-300 pl-4 h-10 w-100&quot;<br>                    name=&quot;message&quot; placeholder=&quot;chat here&quot; /&gt;<br>                &lt;button ws-send<br>                    class=&quot;absolute flex items-center justify-center h-full w-12 right-0 top-0 text-gray-400 hover:text-gray-600&quot;&gt;<br>                    &lt;svg class=&quot;w-4 h-4 transform rotate-45 -mt-px&quot; fill=&quot;none&quot;<br>                        stroke=&quot;currentColor&quot; viewBox=&quot;0 0 24 24&quot;<br>                        xmlns=&quot;http://www.w3.org/2000/svg&quot;&gt;<br>                        &lt;path stroke-linecap=&quot;round&quot; stroke-linejoin=&quot;round&quot; stroke-width=&quot;2&quot;<br>                            d=&quot;M12 19l9 2-9-18-9 18 9-2zm0 0v-8&quot;&gt;&lt;/path&gt;<br>                    &lt;/svg&gt;<br>                &lt;/button&gt;<br>            &lt;/form&gt;</pre><p>Notice at the button tag I have added the <strong>ws-send </strong>htmx attribute, this attribute will send a message to the nearest WebSocket based on the trigger value. Since it is a button, the trigger value will always be onClick event or when you press the Enter key on your keyboard. Htmx will send the message in JSON format and that’s why in the Consumer’s receive method we have to deserialize the text_data sent by the client. When Htmx sends the message to the server via the WebSocket, the <strong>receive </strong>method in the consumer class will get the message and save it in the database then it will retrieve all the messages in that room and send it back to the client in a HTML format.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd91faf5a4c622a3b9d4b1fa743563dd/href">https://medium.com/media/bd91faf5a4c622a3b9d4b1fa743563dd/href</a></iframe><p>This will be the response sent to the client by the server. Take note of the following htmx attribute found in line 14 of the code snippet above <strong>hx-swap-oob=” true”. </strong>This will prompt the htmx to swap all the elements with the same ID as that of all elements in the response and by using this mechanism that’s how a notification is sent to the user because there’s an empty notification element that gets swapped by the responses in the server. This process enables us to change the input tag and render all our chats because the response sends new elements that swap the current elements in the DOM. When you look at our <strong>chats.html </strong>file. Elements that have IDs are swapped by new elements in the response which have the same ID value.</p><p><strong>Conclusion</strong></p><p>Our consumer class is synchronous, It is possible to make it asynchronous by using Python asyncio. To make asynchronous consumers you have to create a consumer class that inherits from <strong>AsyncWebsocketConsumer. </strong><a href="https://channels.readthedocs.io/en/latest/tutorial/part_3.html">Read more</a></p><p>Htmx is an awesome library and it integrates well with the Django template framework because it uses HTML elements. Htmx helps you to do more with less.</p><p><strong>Resources.</strong></p><ol><li>htmx documentation</li></ol><p><a href="https://htmx.org/docs/#swapping">Documentation</a></p><p>2. Django documentation</p><p><a href="https://docs.djangoproject.com/en/4.2/intro/tutorial01/">Django</a></p><p>3. Django channels documentation</p><p><a href="https://channels.readthedocs.io/en/latest/tutorial/part_1.html">Tutorial Part 1: Basic Setup - Channels 4.0.0 documentation</a></p><p>4. Docker with Redis</p><p><a href="https://www.docker.com/blog/how-to-use-the-redis-docker-official-image/">How to Use the Redis Docker Official Image</a></p><p>5. Project GitHub</p><p><a href="https://github.com/chrisgathuthi/connect.git">GitHub - chrisgathuthi/connect</a></p><p>7. Further reading</p><p><a href="https://testdriven.io/blog/django-channels/#what-is-django-channels">Introduction to Django Channels</a></p><p>8. WebSockets</p><p><a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API">WebSocket API (WebSockets) - Web APIs | MDN</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd8174f59a03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Asynchronous form field validation with HTMX and Django.]]></title>
            <link>https://medium.com/@soverignchriss/asynchronous-form-field-validation-with-htmx-and-django-eb721165b5e8?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb721165b5e8</guid>
            <category><![CDATA[django-rest-framework]]></category>
            <category><![CDATA[python-web-developer]]></category>
            <category><![CDATA[htmx]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[django]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Mon, 25 Sep 2023 06:53:14 GMT</pubDate>
            <atom:updated>2023-09-25T06:53:14.781Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q2nn5LnlI6Er5UAzP_WjzQ.png" /></figure><p>HTMX is a library that allows you to access modern browser features directly from HTML, rather than using JavaScript. Developing front-ends of web browsers is now easier and simpler thanks to this new library. For the past few years, front-end development has shifted to a new paradigm of developing user interfaces, a single-page application is widely used. This approach has its fair share of challenges which we will not delve into right now. With htmx, you can use it with any backend templating framework you choose. You don’t have to use Vue, Svelte, or React for your front-end. Django has a feature-rich and powerful templating feature that we can easily incorporate htmx into it and make Ajax requests to the backend. htmx enables us to improve the user experience by making requests without initiating a full-page reload.</p><p>We have endless possibilities with htmx therefore htmx is not limited to only what we will cover. In this article, we will cover how to use htmx to make asynchronous form-field validation before submitting the form.</p><ol><li><strong>Form validation</strong></li></ol><p>When you fill in a form and submit it, Django will validate the form, and if it has any errors it will render the form and show the error message. Well this process works well with Django but we can improve the user experience by making real-time validation, Instead of making validation after you have submitted the form, validation will be made as you fill in the form therefore an Ajax request will be made to the backend which has a form field data and validate it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/727c06d84519820201fa60190f7e598c/href">https://medium.com/media/727c06d84519820201fa60190f7e598c/href</a></iframe><p>The above code snippet is a form class for creating a user registration form instance. In the code, we can see some htmx attribute that has a prefix of <strong>hx-. </strong>The “hx” attributes are used to define the behaviour of how the request-response cycle will take place and which HTML element will be modified. In this particular tutorial, we are going to validate whether the username and email filled in the registration form belong to a user that already exists. If the email already exists an error will be shown under the form input element. How this process will be executed, we will start by defining the form class. In the form class, I have defined the following htmx attributes: h<strong>x-post </strong>This attribute will use htmx to send a post request to the path specified by reverse_lazy, <strong>hx-target, </strong>this attribute takes a valid CSS selector that specifies which HTML element will be replaced by the response sent by <strong>hx-post. hx-trigger, </strong>this attribute takes in a valid trigger event that will initiate the <strong>hx-post </strong>attribute to make an htmx request. In the snippet above the keyup event will initiate the request after the input element has more than three characters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fa75346925f02fe7426ab0ec8da21981/href">https://medium.com/media/fa75346925f02fe7426ab0ec8da21981/href</a></iframe><p>Once we have defined the form class. We have to create the following view functions or classes. I prefer using classes, In the code snippet above, the <strong>HomeView </strong>class is responsible for showing the form page, and receiving form submissions, and saving them. The <strong>UsernameValidation &amp; EmailValidation </strong>class is responsible for validating username and email fields respectively. Remember the <strong>hx-post </strong>attribute, it is responsible for making requests. Those requests will be handled by the validation views. The views will check whether a similar username or email with the same characters exists or not. If it form field fails validation a HTTP response with the error will be shown to the form field. Remember, we defined <strong>hx-trigger, </strong>which happens when the user types in the input field.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cdf87052ab5784fb1c36f2609b814c16/href">https://medium.com/media/cdf87052ab5784fb1c36f2609b814c16/href</a></iframe><p>The code snippet above is from a Django template. We have the form field which is defined using Django template tags syntax. At the email field and username field, there’s <strong>&lt;p&gt; tag. </strong>When we define the <strong>hx-target </strong>in the form, the target defined there is the <strong>&lt;p&gt; </strong>tag id attribute. When htmx makes a request, the response will be swapped inside the <strong>&lt;p&gt; </strong>tag by default. Following that process, we can show field errors while filling in the registration form.</p><p>When there’s an error in the form we have to make sure that the error is corrected before the user submits the form. From my illustrations above a user can still submit a form with an error even though Django by default will perform validation on submission we can prevent the user from submitting the form if it has an error by disabling the submit button. We will do this by modifying the validation response and the form class.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0465944106f606168e0a51265488270e/href">https://medium.com/media/0465944106f606168e0a51265488270e/href</a></iframe><p>The return response of both validation views returns HTML tags of the button and <strong>&lt;p</strong>&gt; tag. When there’s an error, the <strong>&lt;p&gt;</strong> tag will have the error message and the submit button will be replaced with the response button which has a disable attribute therefore there’s no way a user can submit the form if it has an error.</p><p>To selectively pick out a portion of the response and replace it in a different part of the DOM, we will use new HTMX attributes, <strong>hx-select &amp; hx-select-oob. </strong>To use these attributes, I have to modify the form class.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d47c1b42ab3c1c992f516d994262a25/href">https://medium.com/media/8d47c1b42ab3c1c992f516d994262a25/href</a></iframe><p>We have added new HTMX attributes in our form class definition. The <strong>hx-select</strong> attribute is used to pick out a portion of the response and replace it with the element defined by the <strong>hx-target. </strong>In our case, the <strong>hx-target </strong>is a &lt;<strong>p&gt; tag. </strong>The <strong>&lt;P&gt;</strong> tag will be replaced with a new <strong>&lt;p&gt; </strong>tag from the validation response which has the error message. The <strong>hx-select-oob</strong> takes a CSS selector of an element that will be replaced with a response. The target element and the element that will be replaced must have the same selector in our case they both have the same element ID of <strong>#submitBtn. </strong>In simple terms, when you want to replace two elements in the DOM with a HTML response at the same time you use <strong>hx-select &amp; hx-swap-oob. </strong>The hx-select will replace the target element with the selected HTML element and <strong>hx-swap-oob</strong> will replace another element provided they have the same CSS selector.</p><p><strong>References</strong></p><ol><li><a href="https://htmx.org/docs/#oob_swaps">htmx documentation</a></li><li><a href="https://youtu.be/JhskwvJuXF4?list=PL-2EBeDYMIbRByZ8GXhcnQSuv2dog4JxY">Youtube tutorial.</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb721165b5e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding select_related and prefetch_related Methods in Django ORM.]]></title>
            <link>https://medium.com/@soverignchriss/understanding-select-related-and-prefetch-related-methods-in-django-orm-db36003d5fbf?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/db36003d5fbf</guid>
            <category><![CDATA[django-framework]]></category>
            <category><![CDATA[python-web-developer]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Tue, 15 Aug 2023 07:42:05 GMT</pubDate>
            <atom:updated>2023-08-15T07:42:05.780Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VjVO6-6TYcJuFzLNW2alhg.png" /></figure><p>Django web framework has a powerful object-relational mapping (ORM). ORM is a programming technique used in software development to bridge the gap between the object-oriented programming paradigm and relational database. The select_related and prefetch_related methods are used to boost the performance of your database query. To find out why these two methods are performance boosters we have to find out how Django’s ORM queries related objects in the absence of the two methods.</p><p>Consider the following piece of code. It is a block of code for a model that we will use in this article to demonstrate the use case of select_related and prefetch_related methods.</p><pre>class Plan(models.Model):<br><br>    &quot;&quot;&quot;user plan subscriptions&quot;&quot;&quot;<br>    class Packages(models.TextChoices):<br>        NOVICE = &quot;novice&quot;, &quot;Novice&quot;<br>        ELITE = &quot;elite&quot;, &quot;Elite&quot;  # lowercase, check celery task<br>    package = models.CharField(verbose_name=&quot;plan&quot;, max_length=10, choices=Packages.choices,<br>                               default=None)  # Novice, Elite, Legendary<br>    level = models.PositiveIntegerField(default=1)<br>    link = models.URLField(verbose_name=&quot;link&quot;, unique=True, editable=False, null=True)<br>    commission = models.DecimalField(decimal_places=2, max_digits=10, editable=False, default=0.00)<br>    referals = models.ManyToManyField(&quot;Account&quot;, related_name=&quot;invites&quot;)<br>    is_paid = models.BooleanField(verbose_name=&quot;account validity&quot;, default=True, blank=True)<br><br>    def __str__(self):<br>        return self.package<br><br>class Account(models.Model):<br><br>    &quot;&quot;&quot;Registered user accounts&quot;&quot;&quot;<br><br>    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name=&quot;account&quot;)<br>    plan = models.ManyToManyField(Plan, related_name=&quot;subscription&quot;, blank=True)<br><br>    def __str__(self):<br>        return f&quot;{self.user.username}&#39;s account&quot;<br><br>class Blog(models.Model):<br><br>    &quot;&quot;&quot;Blog model&quot;&quot;&quot;<br><br>    account = models.ForeignKey(Account, on_delete=models.SET_NULL, null=True, blank=True)<br>    title = models.CharField(max_length=200)<br>    slug = models.SlugField()<br>    content = models.TextField()<br>    created_at = models.DateTimeField(auto_now=True)<br><br>    def __str__(self):<br>        return self.title</pre><p>Django keeps a cache of each query result in the QuerySet’s cache the first time it is evaluated and in subsequent evaluations of the QuerySet it will reuse the cached results however, it is also important to note that not all evaluated queries are cached. From the code snippet above let&#39;s consider the following query</p><pre># hits the database<br>blog = Blog.objects.get(id=1)<br><br># hits the database to retreive the related account<br>print(blog.account)<br><br># hits the database to get all Blog objects<br>queryset = Blog.objects.all()<br><br># iterating the queryset<br>for qs in queryset:<br>    print(qs.account) # hits the database to retreive all related accounts</pre><p>When iterating through a Blog QuerySet, Django performs two queries to retrieve related objects. It is possible to reduce the number of database hits by using select_related and prefetch_related methods.</p><ol><li><strong>Select_related(*fields)</strong></li></ol><p>This method returns a QuerySet that will “follow” foreign-key relationships, selecting additional related-object data when it executes its query. This is a performance booster that results in a single more complex query but later use of foreign-key or one-to-one relationships won’t require additional database hits. We can optimize the above queries and reduce the number of database hits by introducing the select_related method. It takes a string of the related field attributes. In our case, the relationship is established at the Blog’s account model attribute which is a foreign-key relationship.</p><pre># example 1<br>blogs = Blogs.objects.select_related(&quot;account&quot;)<br>for blog in blogs:<br>    print(blog.account)<br><br># example 2<br>blog = Blogs.objects.select_related(&quot;account__user&quot;).get(id=1)<br>print(blog.account)<br>print(blog.user)<br><br># example 3<br>account = Account.objects.select_related(&quot;user&quot;).get(id=1)<br>print(account.user) # no database hit<br><br></pre><p>From the above snippet of code, we can take a look at several use cases of select_related applied to make efficient queries.</p><p>In example 1, the select_related method is used to fetch all related account objects in a single query note that the account argument in select_related in the field with the Foreign-Key relation therefore if there’s a need to access the related objects from blog instances, Django ORM will not hit the database to query related objects because all related objects were queried initially and stored in the QuerySet’s cache. The method select_related can take the model attribute that has the foreign key defined or the related_name if defined in the foreign key field.</p><p>In example 2, It is possible to query objects that have foreign-key or one-to-one relationships and span multiple models. The query will cache the related Account and User object meaning if the objects are accessed they will be retrieved from the cache and not from the database. It does this by performing a SQL join and fetching the related objects in a single query, rather than issuing separate queries for each related object. Select_related is applied to mitigate a problem referred to as N + 1.</p><p>The N + 1 essentially means you have one main query to retrieve objects and N number of queries to retrieve related objects. As much as the select_related method is a query optimizer you should put in mind that overuse of it or performing complex queries of complex relationships may lead to large and more complex queries and fail to optimize your query. It is recommended for more complex relationships to use the prefetch_related method.</p><p><strong>2. Prefetch_related(*lookup)</strong></p><pre># example 1 <br>accounts = Account.objects.prefetch_related(&quot;plan&quot;)<br> for account in accounts:<br>    print(account.user)<br>    for plan in account.plan.all():<br>        print(plan.package)<br><br># example 2<br>accounts = Account.objects.prefetch_related(&quot;blog_set&quot;)<br>for account in accounts:<br>    print(account.user)<br>    for blog in account.blog_set.all():<br>        print(blog.title)<br><br></pre><p>The prefetch_related method returns a QuerySet that will automatically retrieve, in a single batch, related objects for each of the specified lookups. This has a similar purpose to select_related, in that, both are designed to stop the deluge of database queries that are caused by accessing related objects, but the strategy is quite different.</p><p>select_related works by creating an SQL join and including the fields of the related object in the <strong>SELECT</strong> statement. For this reason, <strong>select_related</strong> gets the related objects in the same database query.</p><p>In example 1, in the snippet of code above, we are prefetching related objects in a Many-To-Many field. The account model has a plan attribute that defines Many-To-Many relationships with the Plan model. We are looping over all account instances while printing related user objects. In the inner loop, we are printing the plan package name while iterating through all plan instances of each individual account of the outer loop. Account instances will only be queried once without triggering additional queries during related object iteration.</p><p>In example 2, prefetch_related is used in the reverse relationship of account and blog. prefetch_related method takes a reverse manager as the lookup. Prefetch_related is ideal for the reverse relationship because it will perform fewer queries.</p><p><strong>How prefetch_related works</strong></p><p>Prefetch_related fetches multiple instances of related objects through a combination of database queries and in-memory processing. The following is the process used by prefetch_related methods to optimize a query.</p><p><strong>a</strong>. <strong>Initial query — </strong>it fetches the main objects using a single query</p><p><strong>b</strong>. <strong>Additional query — </strong>It generates additional queries to fetch related for all the main objects in a single batch. This helps minimize the number of database round-trips.</p><p><strong>c. In-memory processing</strong> — After retrieving the related objects from the database, Django performs in-memory processing to associate the related objects with the main objects. This can involve creating dictionaries or lists that map main objects to their related objects.</p><p><strong>The difference between prefetch_related and select_related methods</strong></p><p>select_related and prefetch_related may seem similar however, it is not, but both methods are used to optimize queries. The following are some of the distinguishing factors between the two.</p><p><strong><em>a. Type of relationships</em></strong></p><p>prefetch_related — Best suited for optimizing queries involving reverse relationships (e.g., ForeignKey, OneToOneField, ManyToManyField).</p><p>select_related — Optimizes queries involving forward relationships (i.e., ForeignKey fields on the current model).</p><p><strong><em>b. Number of queries</em></strong></p><p>prefetch_related — can still result in multiple queries, especially when dealing with many-to-many relationships or complex queries but typically it will result in fewer queries as compared to when the method is not used</p><p>select_related — Aims to reduce the number of queries to 1 or a small number, as it uses SQL joins to fetch related data in a single query.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db36003d5fbf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Querying Related Objects in Django Web Framework]]></title>
            <link>https://medium.com/@soverignchriss/querying-related-objects-in-django-web-framework-3aece5bd927d?source=rss-318880a6cb12------2</link>
            <guid isPermaLink="false">https://medium.com/p/3aece5bd927d</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[python-web-developer]]></category>
            <category><![CDATA[django]]></category>
            <dc:creator><![CDATA[Christopher Gathuthi]]></dc:creator>
            <pubDate>Wed, 26 Jul 2023 08:29:40 GMT</pubDate>
            <atom:updated>2023-07-26T08:29:40.656Z</atom:updated>
            <content:encoded><![CDATA[<p>According to a Django developer survey conducted in the year 2022 showed that Django models are developers’ favourite core component among respondents. In Django models, we can create relationships between models. Django has the following relationships; one-to-one, one-to-many and many-to-many. In this article, I am going to cover how to query a table that spans multiple relationships.</p><p>In this article, I will assume that you know how to set up a Django project otherwise, you can follow this link to read more about it.</p><p><a href="https://docs.djangoproject.com/en/4.2/intro/install/">setting up a Django project</a></p><p>Consider the following models below.</p><pre>from django.db import models<br>from django.contrib.auth.models import User<br><br># Create your models here.<br><br>class Plan(models.Model):<br><br>    class Packages(models.TextChoices):<br>        NOVICE = &quot;novice&quot;, &quot;Novice&quot;<br>        ELITE = &quot;elite&quot;, &quot;Elite&quot;  <br>    package = models.CharField(verbose_name=&quot;plan&quot;, max_length=10, choices=Packages.choices,<br>                               default=None)  <br>    level = models.PositiveIntegerField(default=1)<br>    link = models.URLField(verbose_name=&quot;link&quot;, unique=True, editable=False, null=True)<br>    commission = models.DecimalField(decimal_places=2, max_digits=10, editable=False, default=0.00)<br>    referals = models.ManyToManyField(&quot;Account&quot;, related_name=&quot;invites&quot;)<br>    is_paid = models.BooleanField(verbose_name=&quot;account validity&quot;, default=True, blank=True)<br><br>class Account(models.Model):<br><br>    user = models.OneToOneField(User, on_delete=models.CASCADE, related_name=&quot;account&quot;)<br>    plan = models.ManyToManyField(Plan, related_name=&quot;subscription&quot;)<br><br>class Blog(models.Model):<br><br>    account = models.ForeignKey(Account, on_delete=models.SET_NULL, null=True, blank=True)<br>    title = models.CharField(max_length=200)<br>    slug = models.SlugField()<br>    content = models.TextField()<br>    created_at = models.DateTimeField(auto_now=True)<br><br></pre><p>All three relationship types are used in the models above. A one-to-one relationship exists between the Account and the User model, Many-To-Many exists between Plan and Account, and Many-To-One exists between Blog and Account.</p><ol><li><strong>One-To-One Relationship</strong></li></ol><p>A one-to-one relationship is a type of entity association in the database where one record in an entity is related to at most one record in the other entity. In our case, the <strong>User </strong>model is related to the <strong>Account </strong>model.</p><p><strong><em>forward relationship</em></strong></p><p>From the <strong>Account </strong>model to the <strong>User </strong>model.</p><pre># return the related user instance<br>account = Account.objects.last()<br>user = account.user.username<br># print the username <br>print(user)</pre><p>From the code snippet above we are displaying the username of the last object in the Account’s query set. How to navigate the forward relationship you first start by creating the instance of the related object in our case it is the Account model and then using the dot notation (.) you can access the attributes of the parent model by using the related attribute (the one with the relationship definition).</p><p><strong><em>backward relationship</em></strong></p><p>In Django ORM, a reverse relationship is automatically created using a reverse relationship manager. By default, this reverse relationship is named with the lowercase name of the model followed by “_set”. However, you can customize this default behavior using the related_name field option. In our case, the related_name is <strong>account</strong>.</p><pre>user = User.objects.last()<br>user = user.account<br># print account object<br>print(user)</pre><p><strong>2. Many-To-One Relationship</strong></p><p>In a <strong>many-to-one </strong>relationship is where one record is associated with more than one record of another entity. Some sources many refer to this relationship as <strong>One-To-Many, </strong>however, it means the same thing</p><p><strong><em>forward relationship</em></strong></p><pre>blog = Blog.objects.last()<br># related account object<br>account = blog.account<br>print(account)<br><br># username of the blog&#39;s account<br>username = blog.account.user.username<br>print(username)</pre><p>Navigating the forward relationship entails using the related field attribute and using the dot notation to go back and get the parent’s attributes from the child.</p><p><strong><em>backward relationship</em></strong></p><pre>account = Account.objects.last()<br><br># queryset of blogs related to account<br>blogs = account.blog_set.all()<br>print(blogs)<br></pre><p>The reverse of the Many-To-One relationship is expected to return a query set unlike the relationship of One-To-One, a default reverse manager is created using the related models&#39; name in lowercase then concatenate <strong>_set i.e Foo_set </strong>or in our case, it will be <strong>blog_set. </strong>It is possible to customize the reverse manager by specifying the <strong>related_name </strong>in the <strong>ForeingKey</strong> field. The reverse manager has various methods that can be used to handle related objects.</p><pre><br>account = Account.objects.last()<br><br>blogs = account.blog_set.all()<br><br># return a queryset of related blog<br>print(blogs)<br><br># create()<br>updated_blogs = account.blog_set.create(title=&quot;Django project&quot;, slug=&quot;Django-project&quot;, content=&quot;Django is an awesome tool&quot;)<br>print(updated_blogs)<br><br># remove()<br>blog = account.blog_set.get(id=1)<br>account.blog_set.remove(blog)<br><br># clear()<br>account.blog_set.clear()</pre><p>I. <strong>create() </strong>This method creates a new object and puts it in the related object set.</p><p>II. <strong>remove() </strong>This method is used to remove a related object from the related object set.</p><p>II. <strong>clear()</strong> This method removes all objects from a related object set</p><p><strong>3. Many-To-Many relationship</strong></p><p>A many-To-Many relationship is a type of association where multiple records in one table are associated with many records in another entity(table). In the example above, there are two many-to-many relationships. We are going to use the one defined in Plan’s model for our illustration.</p><p><strong><em>forward relationship</em></strong></p><pre>account = Account.objects.first()<br><br>plans = account.plan.all()<br><br># queryset of related plans<br>print(plans)</pre><p>The <strong>plan </strong>attribute in the account’s model which has the relationship defined it’s a <strong>ManyRelatedManager </strong>therefore it is used with some query set methods to return a query. From the code block above we have retrieved the first account object then we used the <strong>all() </strong>method to query all related objects from the <strong>Plan</strong> model.</p><p><strong><em>reverse relationship</em></strong></p><pre># retrieve the first plan<br>plan = Plan.objects.first<br><br># reverse relationship<br>accounts = plan.subscription.all()<br><br># queryset of related accounts<br>print(accounts)</pre><p>Making a reverse relationship is similar to the one of Many-To-One, you first create an object of the parent then use the reverse manager to access the child model. In our case, the related manager is the subscription. The reverse manager also accepts methods for handling related objects. You can <a href="https://docs.djangoproject.com/en/4.2/topics/db/queries/#additional-methods-to-handle-related-objects">check</a> here for more related methods.</p><p><strong>References</strong></p><ol><li><a href="https://github.com/chrisgathuthi/orm.git">Github</a></li><li><a href="https://docs.djangoproject.com/en/4.2/topics/db/queries/#related-objects">Django Official Documentation</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3aece5bd927d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>