<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Philamer Sune on Medium]]></title>
        <description><![CDATA[Stories by Philamer Sune on Medium]]></description>
        <link>https://medium.com/@philamersune?source=rss-ddad1b6ed626------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*sfavaH8R1-qXbwe3.</url>
            <title>Stories by Philamer Sune on Medium</title>
            <link>https://medium.com/@philamersune?source=rss-ddad1b6ed626------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 03:06:19 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@philamersune/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Fixing “SSL error: decryption failed or bad record mac”]]></title>
            <link>https://medium.com/@philamersune/fixing-ssl-error-decryption-failed-or-bad-record-mac-d668e71a5409?source=rss-ddad1b6ed626------2</link>
            <guid isPermaLink="false">https://medium.com/p/d668e71a5409</guid>
            <category><![CDATA[python3]]></category>
            <category><![CDATA[django]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[security]]></category>
            <dc:creator><![CDATA[Philamer Sune]]></dc:creator>
            <pubDate>Sun, 31 May 2020 15:27:36 GMT</pubDate>
            <atom:updated>2020-05-31T15:27:36.734Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NuofsHWYi-C4PPA5s_63lQ.png" /></figure><p>A couple of days ago, I was greeted by a message that a particular API endpoint in our application is returning 500 Server Error. I was curious because I had tests for that and even tried calling them manually a couple of times when I was coding. I logged in and immediately saw this error django.db.utils.OperationalError: SSL error: decryption failed or bad record mac for a line number that queries the database. I grabbed the terminal and tried python manage.py shell if I can get results back from my models. Ha! I could! This is weird.</p><p>The affected part in the application is where the system updates the user’s registration tier. There are three steps in the registration process of our system:<em> </em><strong><em>(1)</em></strong><em> user registered and will be assigned </em><em>tier=initial_reg </em><strong><em>(2)</em></strong><em> user updates profile filling up important information and is assigned </em><em>tier=pending </em><strong><em>(3)</em></strong><em> admin verifies the submitted information and if approved will be assigned </em><em>tier=full_reg</em>. All other endpoints that queries the database works properly and I’m starting to scratch my balls that it only happens when admin approves the registration. It also doesn’t help that the endpoint in issue is working correctly in my local Docker development setup. After a lot of pdb and serious keyboard banging, I finally found the issue.</p><p>When admin approves the registration, the system generates a Membership ID; angels will sing, signifying a revered divine event in the users life within the app. The ID is in a specific format and unfortunately I can’t use UUID. The generated ID is then saved in the database and if an IntegrityError occurs, it generates another one until it can be saved successfully. Membership ID generation will then block the request and for this reason I implemented a timeout() utility method which spawns a new process and times out at a specified max seconds. This is where it all goes awry.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8837588619ba690406742ef161a5a863/href">https://medium.com/media/8837588619ba690406742ef161a5a863/href</a></iframe><p>The SSL error: decryption failed or bad record mac occurs either when the certificate is invalid or the message hash value has been tampered; in our case it’s because of the latter. Django creates a single database connection when it tries to query for the first time. Any subsequent calls to the database will use this existing connection until it is expired or closed, in which it will automatically create a new one the next time you query. The PostgreSQL engine in Django uses psycopg to talk to the database; according to the <a href="https://www.psycopg.org/docs/usage.html#thread-safety">document</a> it is level 2 thread safe. Unfortunately, the timeout() method is using multiprocessing module and therefore tampers the SSL MAC. There are different ways to fix this. We can either<em> </em><strong><em>(1)</em></strong><em> use basic threads instead of spawning a new process or </em><strong><em>(2)</em></strong><em> use a new database connection in the </em><em>timeout() method. We can also </em><strong><em>(3)</em></strong><em> scrap the </em><em>timeout() method altogether and handle the async task properly via Celery.</em></p><p>The timeout() method is simple, generic, and a quick implementation to limit the execution time of a method. We can use this for anything that takes time, not only for those that queries databases. I think most of the time spawning a new process is safer than using threads. At this point in time, I also don’t want to introduce and maintain another moving part like Celery yet; so we’re choosing to use a new database connection to fix the issue! Unfortunately, it seems it wouldn’t be as pretty. In order to create a new database connection, you just have to close the existing one and Django will create a new one. But this doesn’t work if you are inside a database transaction — it will fail InterfaceError: connection already closed. There’s also no easy way to pass a new database connection when using the models. The closest would be the .using() method where you specify a database alias. You can probably overload the cached property django.db.connections.databases and add your alias with new connection but I wouldn’t really do that! Haha! So we’re left with executing custom SQL directly:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/80ddd87d46fde56bd828ff755beafe53/href">https://medium.com/media/80ddd87d46fde56bd828ff755beafe53/href</a></iframe><p>The issue we talked about could probably be detected earlier if I just use PostgreSQL, like in Production, instead of SQLite in memory when running the integration tests. I also could’ve tested it works properly in Dev environment where it uses proper PostgreSQL. Silly me! Overall, the debugging process have kept me busy in this quarantine period while working at home. I think it’s also a little fun and a good learning experience for becoming a better Software Engineer.</p><p>Keep safe folks! Let’s come back stronger after this pandemic.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d668e71a5409" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bypassing the GIL: A concurrent Kafka Consumer]]></title>
            <link>https://medium.com/@philamersune/bypassing-the-gil-a-concurrent-kafka-consumer-dcc44ae9260a?source=rss-ddad1b6ed626------2</link>
            <guid isPermaLink="false">https://medium.com/p/dcc44ae9260a</guid>
            <category><![CDATA[event-driven-systems]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Philamer Sune]]></dc:creator>
            <pubDate>Sun, 26 Apr 2020 10:40:55 GMT</pubDate>
            <atom:updated>2020-04-26T14:33:57.583Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GfpHk0gZOddAJKrER86i4g.png" /></figure><p>Most <a href="https://martinfowler.com/articles/201701-event-driven.html">event-driven systems</a> are using Kafka as a backbone service to serve their event-driven architecture. Kafka is a high performant, highly scalable event streaming platform — it is capable of handling a trillion messages a day! If you have tried processing HTTP requests asynchronously, you must be familiar with RabbitMQ. Kafka and RabbitMQ have different use cases but simply think Kafka is like RabbitMQ with steroids.</p><p>Usually, an app publishes a message to a Kafka topic and a consumer fetches that message via the topic name. In a distributed environment, several instances of your app might publish several messages in the topic. If your consumer can’t keep up processing these messages, they might be discarded depending on your cluster’s message retention setting. You need multiple consumer workers. You can do this simply by running another consumer instance specified by <a href="https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md">group.id</a>. But you can only run as many consumer instances as Kafka partitions you have. Otherwise, the excess consumers will just sit idly until one of the other consumers die. And you can’t also just set 1000 partitions, for example, since there are <a href="https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster/">potential impacts</a> you need to be aware of. You need a multiprocess multithreaded consumer.</p><p>You probably have used either uWSGI or Gunicorn <em>(or both)</em>. Running these web servers, you can specify the number of --processes and number of --threads it can spawn to respond to requests. The same thing is what we want to achieve with our concurrent Kafka consumer. We want to be able to identify the number of processes which consume messages to each of our topic’s partitions and number of threads which can work in processing the consumed message. With this approach, <a href="https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html">we are separating <strong>consumption</strong> from <strong>processing</strong></a> and therefore bypassing the GIL. Spawning processes is resource-heavy but not limited by the GIL and we take advantage of it by spawning only as much partitions as we have. Threads, on the other hand, are lightweight and are limited by GIL but they are still useful for I/O-bound tasks A.K.A processing the message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*yzXpPAKQHTGClw-wsWiKRA.png" /><figcaption>Kafka consumer model from partitions, processes, to threads</figcaption></figure><p>We’re going to use <a href="https://github.com/confluentinc/confluent-kafka-python">confluent-kafka-python</a> to build our consumer. There are several good Kafka Python clients you can find like <a href="https://github.com/dpkp/kafka-python">kafka-python</a> and <a href="https://github.com/Parsely/pykafka">pykafka</a>. But the main reason I choose Confluent’s is because they are founded by the same team that created Apache Kafka and therefore should be future-proof. Under the hood, it’s also a wrapper for <a href="https://github.com/edenhill/librdkafka">librdkafka</a> which is a C library for the Apache Kafka protocol that can exceed consuming three million messages per second! This GitHub <a href="https://github.com/Parsely/pykafka/issues/334">issue</a> shows more explanation about popular Python Kafka clients.</p><p>Below is our concurrent Kafka consumer:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8411075b4f89ddbf00de9d4b322f80b4/href">https://medium.com/media/8411075b4f89ddbf00de9d4b322f80b4/href</a></iframe><p>In the code above, we pass the list of <a href="https://kafka.apache.org/documentation/#configuration">Kafka settings</a> via <em>kafka_kwargs</em>. Note that we don’t need to pass all the list of servers inboostrap.servers setting. At least two is ok as this is only used to get cluster metadata. On the main() function, we are spawning processes specified by num_workers setting. We also check the count of alive processes and automatically spawn new ones in case it doesn’t match num_workers <em>(i.e. a process was terminated)</em>. Setting daemon=True to multiprocessing.Process() makes the child process daemonic and will not be able to spawn child process of its own. Daemonic processes are also terminated automatically if no non-daemon process exists <em>(e.g. the main program exits)</em> which what we want. Inside _consume() function, we specify a FIFO queue.Queue() object with a size equal to num_threads setting. If the queue is full, q.put() will block until one of the other messages have been processed. Finally, we _process_msg() by simply printing the message and commit to Kafka to increase the offset in that partition. We commit manually instead of setting enable.auto.commit=True because we will lose the message if one of the threads weren’t able to process successfully. The time.sleep() is for simulating I/O.</p><p>In summary, we were able to create a concurrent Kafka consumer not limited to the number of partitions. We took advantage of multiprocessing but not abused it to process the message in parallel. Instead, we use threading for concurrency which is best suited for I/O-bound tasks which most of the time is what we do with the message <em>(e.g. insert to database, send to API service, etc.)</em>. Ultimately, we have bypassed the GIL by using the right Python concurrency module for the task!</p><p>Enjoy coding con…crr…unt…yel!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dcc44ae9260a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using PostgreSQL JSONField in SQLite]]></title>
            <link>https://medium.com/@philamersune/using-postgresql-jsonfield-in-sqlite-95ad4ad2e5f1?source=rss-ddad1b6ed626------2</link>
            <guid isPermaLink="false">https://medium.com/p/95ad4ad2e5f1</guid>
            <category><![CDATA[django]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[postgres]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[sqlite]]></category>
            <dc:creator><![CDATA[Philamer Sune]]></dc:creator>
            <pubDate>Mon, 25 Nov 2019 13:16:22 GMT</pubDate>
            <atom:updated>2019-11-25T13:16:22.835Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RXLFZjatTu3tBsOwWKq9xA.png" /></figure><p>I think we can all agree that PostgreSQL is an efficient, solid, production-grade RDBMS. As of <a href="https://www.postgresql.org/docs/9.4/release-9-4.html">PostgreSQL 9.4</a>, JSON support has been improved by having extractor/constructor methods and storage as JSONB. It is almost having both RDBMS and NoSQL in one DB system. SQLite, on the other hand, is a simple RDBMS which saves data either in file or memory. It is almost always used as a throw away DB for running local development server or tests. Unlike PostgreSQL, JSON is not natively supported in SQLite.</p><p>I am so used to running my integration tests in SQLite via memory. I don’t have to create “setup” and “teardown” functions because the test data are gone after the tests run. I hit this block when I started using <a href="https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#jsonfield">Django PostgreSQL specific JSONField</a>. <em>What if I save JSON as </em><strong><em>“text”</em></strong><em> in SQLite and </em><strong><em>“json.loads”</em></strong><em> the value when I fetch?</em></p><p>Django ORM, or any other ORM, provides a way to <a href="https://docs.djangoproject.com/en/2.2/howto/custom-model-fields/">create a custom Model field</a>. Basic functions to implement are <strong>“to_python()”</strong> which converts DB value to Python object and <strong>“get_prep_value()”</strong> which converts Python object to DB value. <a href="https://github.com/coleifer/peewee/blob/3.12.0/playhouse/sqlite_ext.py#L122">See implementation in Peewee ORM</a>.</p><p>Below is our implementation for Django ORM.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/90dd8026c2b826a71ffcd88b490c1931/href">https://medium.com/media/90dd8026c2b826a71ffcd88b490c1931/href</a></iframe><p>We can use the script by saving as <strong>&quot;&lt;app_name&gt;/models/fields.py”</strong>. In here, we specify same class name for our custom SQLite Model fields and Django PostgreSQL fields. It’s important that we define only one of these depending on which DB Engine is configured. <strong>Otherwise, the migration file will import the exact class for that DB and make the migrations work only for that specific DB Engine</strong>.</p><p>Also note that filtering by JSON key is not supported in this implementation <em>(ie. </em><em>.filter(&lt;json_column&gt;__&lt;json_key&gt;=&lt;value&gt;))</em>. But may be possible by registering a <a href="https://docs.djangoproject.com/en/2.2/ref/models/lookups/">custom lookup</a>.</p><p>This implementation of a custom JSONField for SQLite is very raw. It works on basic purposes. For advance requirements, always see the documentation of your favorite ORM. RTFM! There should be more things you can tweak.</p><p>Have fun hacking!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=95ad4ad2e5f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Docker images with Drone CI faster but less furious]]></title>
            <link>https://medium.com/@philamersune/building-docker-images-with-drone-ci-faster-but-less-furious-e3ee5b7adbc0?source=rss-ddad1b6ed626------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3ee5b7adbc0</guid>
            <category><![CDATA[drone-ci]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Philamer Sune]]></dc:creator>
            <pubDate>Sun, 07 Apr 2019 02:51:24 GMT</pubDate>
            <atom:updated>2019-04-07T02:51:24.676Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OS7KV8bvG8v52h8B00dOcA.png" /></figure><p>If you’re looking for a simple yet extensible open-source CI platform, you can stop looking now and go to <a href="https://drone.io/">https://drone.io</a>. Drone CI is fairly new platform to power your Continuous Delivery workflow. It’s open-source, container-native, extensible with pre-made plugins, and very easy to setup. In fact, one of the things that will get you furious immediately after setup is Docker layer caching.</p><p>Caching in Drone isn’t handled natively but can be achieved through custom plugins. There are plugins to cache artifacts in AWS S3 (plugins/s3-cache), GCP Cloud Storage (homerovalle/drone-gcs-cache), Host Volume (drillster/drone-volume-cache), and even with External Server via SFTP (appleboy/drone-sftp-cache). The problem arises when you want to build an image and push it to registry using DinD (Docker in Docker). To build and push image in AWS ECR, for instance, you’ll use plugins/ecr which uses DinD internally. Since it&#39;s inside docker-ception, artifacts created during the build step isn&#39;t easily available. What&#39;s available are the Docker layers which you can probably cache using the plugins but I don&#39;t think is a great idea. Fortunately, there is a better way! Starting with Docker 1.13, docker build command can accept --cache-from argument which specifies a tagged image as cache source. This will pull the image of the specified cache source and only build the layers from bottom until the end of the build when it is different. You might be thinking that this is awesome. It is. But be mindful of some caveats:</p><ul><li>Since it will pull the cache source image, make sure it is light in bytes. You might end up furious about the time pulling it.</li><li>If you end up changing something that causes the bottom layer to update, specifying a cache source might have no impact.</li></ul><p>In addition, you can also change the Storage Driver used by DinD for lesser overhead in I/O. The Storage Driver in Docker manages how the layers read/write files. Best way to choose Storage Driver used by DinD is to match it with the host. You can check via docker info in the host machine. Most probably it will not match the default value for DinD which is vfs. The vfs Storage Driver does some overhead because it deep-copies files from the Docker layers to the writable-layer; but doing so makes it compatible with most environments (e.g. Windows, Linux, Mac). For Linux machines, overlay2 is usually used.</p><p>Right now, you might be looking at your .drone.yml file and wondering &quot;how the hell do I specify the cache source image and Storage Driver?&quot;. Be less furious. If you are pushing to Docker Hub and is using plugins/docker you can specify the storage_driver setting and then set the environment variable PLUGIN_CACHE_FROM to your cache source image. On the other hand, if you are pushing to AWS ECR or other registry, there might be no storage_driver setting; instead, you have to set PLUGIN_STORAGE_DRIVER environment variable. Drone CI is fairly new and these things aren&#39;t documented yet. But it&#39;s open-source and checking it is a good practice. :)</p><p>The rest of this post shows sample configuration and before-and-after build times. Have fun droning!</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f94de971a730f46f3c1fbc08def81e8/href">https://medium.com/media/0f94de971a730f46f3c1fbc08def81e8/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K40y87VJqJq1aZoK8uS-9g.png" /><figcaption>Before-and-after build and push times to AWS ECR —<strong> from ~4m to ~1m</strong></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3ee5b7adbc0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>