<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Berty Thomas on Medium]]></title>
        <description><![CDATA[Stories by Berty Thomas on Medium]]></description>
        <link>https://medium.com/@bertythomas?source=rss-3c2f90b8af70------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*YOnT2ahHrSGZIkiMGvsr8Q.jpeg</url>
            <title>Stories by Berty Thomas on Medium</title>
            <link>https://medium.com/@bertythomas?source=rss-3c2f90b8af70------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:11:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bertythomas/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The tech behind the under45.in vaccination alerts project]]></title>
            <link>https://medium.com/@bertythomas/the-tech-behind-the-under45-in-vaccination-alerts-project-3037a8e31d64?source=rss-3c2f90b8af70------2</link>
            <guid isPermaLink="false">https://medium.com/p/3037a8e31d64</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[covid-vaccine-update]]></category>
            <category><![CDATA[telegram]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[covid-vaccination-india]]></category>
            <dc:creator><![CDATA[Berty Thomas]]></dc:creator>
            <pubDate>Sat, 20 Aug 2022 14:04:16 GMT</pubDate>
            <atom:updated>2022-08-22T14:52:51.864Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEsheQFSU_CUXE49577dFQ.png" /></figure><p>At the very outset, let me put this disclaimer. If you are looking for some breakthrough piece of engineering, I think you may get disappointed. If you want to understand the thought process behind a hacky piece of code that proved extremely useful for millions of people, read on.</p><p>I know it has been more than a year since I wrote the last <a href="https://medium.com/@bertythomas/7-important-factors-that-led-to-the-success-of-under45-in-8abe2621c17e">article on the under45project</a> where I mentioned that I would reveal the tech stack behind it. Apologies for the delay.</p><p>During the peak of the Covid second wave in India, we saw the start of vaccination for the 18–45 age group. With the supplies limited and the demand going through the roof, everyone was trying to grab a vaccination slot on the CoWIN platform. The under45 project intended to alert anyone who subscribed to the relevant Telegram channel as and when a vaccination slot was available.</p><p>Things got difficult once the Govt. introduced rate limits to the CoWIN API. The limit introduced was <em>100 API calls per 5 minutes per IP</em>. To make matters worse, the <em>CalendarByDistrict</em> API, which provides 7day slot data was responding with data that was 5min behind, rendering it useless. So the only API that we could use was the <em>FindByDistrict</em> API, which gives data only for the specific day. This means that we had to call this API 7 times to get a week’s worth of slot data which the <em>CalendarByDistrict</em> API was providing, hence magnifying our problem <strong>7 times</strong>! So this article will only talk about the architecture that was used after those rate limits were put in place (it was too easy prior to those limits).</p><h3>Can’t we use proxies?</h3><p>Of course, this is the first thought that comes to any programmer’s mind when it comes to IP based rate limits. However, there were a few issues:</p><ol><li>CoWIN restricts API calls to India based IPs, so only India based proxies can be used.</li><li>Then there is a cost associated with the use of proxies. Remember, this was a hobby project and a self funded one at that!</li><li>Latency! This was the biggest issue. Every API call via a proxy takes much longer to complete than a direct call. And when we are talking about searching slots in 700+ districts, this becomes a deal breaker!</li></ol><h3>Increase the server instances?</h3><p>Technically, adding EC2 server instances or reassiging Elastic IPs were possible solutions, but when you consider the earlier mentioned rate limits and the volume of API calls to be made for an effective pan India vaccination notifier, anyone with experience in AWS would understand, the costs would quickly get out of hand! So that was ruled out.</p><h3>God bless the AWS Lambda</h3><p>The AWS Lambda is a serverless stack. Each Lambda instance will use a particular IP for execution. So once we set the AWS region to Mumbai, we suddenly had a large, workable IP pool with amazing performance (unlike the proxies)!</p><p>Each of the Lambdas were run on NodeJS (because other than PHP, that’s what I know!). The input params were the district ID, the from and to dates. The Lambda would then call the <em>FindByDistrict</em> API for each of those dates, concurrently and then stitch the results together in the response.</p><h3>Districts and their refresh frequency</h3><p>Based on the experience from the previous 2 weeks, we understood that it wasn’t prudent to consider all the districts the same. Some city districts, required very high refresh rates, so that we can alert as soon as a slot was available. These are generally metro city areas like Bengaluru (BBMP), Delhi, Mumbai, Pune, etc. We had identified 25 such districts. These districts were checked every 10sec. All other districts were refreshed once every 30sec.</p><h3>The tech stack</h3><p>Now, let’s get to the beautiful part and talk about how it was done.</p><p>There were 2 EC2 instances that act as the main controllers. These 2 instances did the exact same job and fetch the slots for the same districts (via the Lambdas) albeit a few seconds apart. This was done so that in case one EC2 instance went down (ohh that happens!), the slot refresh does not stop. Only the frequency gets affected. So in such a case, instead of refreshing every 10s, it refreshes every 20s or if it was refreshing every 30s, it will refresh every minute (via the other EC2 instance controller).</p><h3>How many Lambdas?</h3><p>Well, a lot!</p><p>There were a thousand Lambdas in total (all created programatically, of course!). These were allocated into 3 Lambda pools.</p><ol><li>A set of 400 Lambdas were allocated for those 25 districts which required high refresh rate.</li><li>A set of 300 Lambdas were allocated to Controller 1 (EC2 instance #1).</li><li>A set of 300 Lambdas were allocated to Controller 2 (EC2 instance #2).</li></ol><p>Each of these Lambdas were like a stack of cards. The API calls were routed via the Lambda on the top of the stack. The moment we get a rate limit error from CoWIN, that Lambda is sent to the bottom of the stack. The API calls are then routed via the next Lambda that is on top of the stack. And this is repeated.</p><p>The implementation was simple. Each of these Lambda IDs were a MySQL table for the respective 3 pools (Pool 1, 2, 3 as mentioned earlier). The table had just 2 columns: LambdaID and last_used. As soon as the rate limit is hit, the last_used is updated with the current timestamp. So once we fetch the table ordered by the last_used column, the first row is the Lambda that is to be used for API routing.</p><h3>The database</h3><p>This was a simple AWS RDS instance that worked independently of these instances. This instance had details of the Lambda pool, the Districts and the Telegram channel IDs associated with each of them. So if I receive a feedback from any user via Twitter saying, <em>“hey, its increasingly getting difficult to find a slot in District XYZ”</em>, all I need to do is adjust the scraping frequency in the MySQL table. It was that easy!</p><h3>Scheduled restart “Jugaad”</h3><p>If I have to pick one Jugaad that was deployed in the under45 project, this would be the one! For reasons unknown to me, after running for hours, the EC2 instances freeze. The fix was to just restart the instance for it to start working again. But then, it could happen anytime. So this constant monitoring was irking me. I remember the time when the worst case scenario happened. I was out to buy some groceries. I got a notification that EC2 controller instance #1 went down. I continued my shopping thinking that it will only slow down the slot refreshes and I can always come back and restart that instance to bring it back to the normal refresh frequency. Then in another 15min, the EC2 controller instance #2 also went down. Now the slot refresh had completely stopped. I started getting tweets from users saying they are not getting notified and I had to rush back to my laptop to restart the servers!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLbRzK7PP9HAqIaT0sIFrA.png" /></figure><p>This prompted me to write a Jugaad code which would restart the two controller instances (never at the same time) when it has been up for 2hrs or more. While this happens, a third EC2 instance will automatically boot up and take the work of the controller instance that is going to be restarted. Once the restart is complete, the backup instance shuts down. Thus it no longer required me to do any kind of manual interventions or monitoring after this!</p><h3>The cost</h3><p>Many subscribers reached out to me saying they would like to contribute towards the server costs. I turned them down easily. Given that the only AWS usage for me were 1000 Lambdas (they are billed only per usage, which was low. Remember, at a time, only 1 lambda is used. The remaining ones are in the pool, un-billed), 3 EC2 instances, 1 RDS instance, it wasn’t expensive at all. In fact, my entire bill for this period was just around <strong>Rs.4,000 ($50)</strong>! Thank you AWS &amp; Telegram!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3037a8e31d64" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[7 important factors that led to the success of under45.in]]></title>
            <link>https://medium.com/@bertythomas/7-important-factors-that-led-to-the-success-of-under45-in-8abe2621c17e?source=rss-3c2f90b8af70------2</link>
            <guid isPermaLink="false">https://medium.com/p/8abe2621c17e</guid>
            <category><![CDATA[covid-vaccination-booking]]></category>
            <category><![CDATA[covid-vaccination-india]]></category>
            <category><![CDATA[cowin]]></category>
            <category><![CDATA[under45]]></category>
            <category><![CDATA[technology-impact]]></category>
            <dc:creator><![CDATA[Berty Thomas]]></dc:creator>
            <pubDate>Sun, 25 Jul 2021 09:41:20 GMT</pubDate>
            <atom:updated>2022-08-20T14:06:14.854Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*X0bsjcuv0G9trjnoDtTrzA.png" /></figure><p><strong>1. The speed of development:</strong><br>As soon as I found that there was a CoWIN API that I could use to search for slots, I wrote a simple search tool around it and pushed it online on 29th April 2021. The code was extremely simple consisting of a real time call to the API based on the state and district parameters which was selectable by the user. It took me just 3hrs to code it (it was honestly that simple).</p><p><strong>2. The clear need for the tool:</strong><br>So many people were trying to find a vaccination slot and all of them found it difficult to navigate through CoWIN to find one. You had to login in via OTP each time to search for a slot and then go through the entire list of vaccination centres to find something for your age group. Why would someone design CoWIN to show all the centres, even when there are no slots available in them? Why would there be no selectable filters in the UI? People just wanted to know if there was a slot available in their district and the under45 tool solved exactly that problem.</p><p><strong>3. Listening to the users:</strong><br>As soon as people started sharing the tool and more and more users started making use of it, the first question that people had in mind is, does this really work? Because remember, there were very few 18+ slots across the country and when you search in under45, you will most likely see the message that there aren’t any slots. So then it was natural for users to ask if this really works or not.</p><blockquote>That’s when I decided to have a history page.</blockquote><p>When users select for each district, if they find a slot, I wrote a script that will capture that district details and the time stamp on which the search was made and show that up in the history. I then linked this page to the homepage saying “where are the slots?”. This allowed people to cross check if ever they felt that this tool doesn’t really work just because the district they searched for didn&#39;t throw up any results. And once they understood that the tool was functional, they shared widely with their friends. Because everyone was looking for slots across the country!</p><blockquote>Lessons from the server crash</blockquote><p>By 1st May 2021, the traffic on the site had increased substantially. I was no specialist in this area and hence I was finding it increasingly difficult to keep the server up and running. I increased the server capacity but I knew even that wouldn’t suffice soon enough.</p><p>That’t when I tried to understand what the users were doing on my site. Most of them were just refreshing the result page to check whether the slots have opened or not. That was fine earlier, but with the ever increasing traffic, that’s not a sustainable model for me. I also started getting feedback from users to create alerts.</p><blockquote>Alerts became inevitable.</blockquote><p>So based on the feedback and the struggle to keep up to the ever increasing load on the server, I knew that the alerts had to be brought it if I had to continue with this.</p><p><strong>4. Using the right tools:</strong></p><p>There were the following options that I had thought of:</p><p>Email<br>Email was ruled out because I knew it would take time to send thousands of emails and speed was the name of the game when it came to these alerts. And getting these emails to people’s inbox (and not spam) was also not an easy thing to do.</p><p>SMS<br>SMS was costly. I didn&#39;t want to take up that costs at all. I didn’t want to make it paid or even ask for donations for this.</p><p>WhatsApp<br>WhatsApp requires a business account to integrate with the API and I was not familiar with that API.</p><p>Telegram<strong><br></strong>Telegram was perfect for me. I didn’t have to ask users for their private data (Name, email, phone number). I had a no-cost option to send the alerts immediately to everyone. There was a one-way mass communication option using Telegram Channels that people could subscribe to. And I was also familiar with the Telegram API. It was the perfect solution for these alerts.</p><p>So I created the channels for all major districts in the country and started pushing the alerts to these channels. I also started widely asking people to use the alerts instead of the website search tool.</p><p><strong>5. Embracing change</strong></p><blockquote>It was time to kill the search.</blockquote><p>Even after creating the alerts, I still had the search tool running on the website which people could use. After CoWIN changed the API limits on the public API, it started hitting those limits pretty quickly. I had the choice to either fix it or just kill the search altogether. I chose the latter and focus completely on the Telegram alerts.</p><p>With the new CoWin API limits in place, I faced another dilemma. Make the cities alert faster or expand to more districts in the country. I couldn’t in all fairness make a choice like this. I had received multiple requests from my current Telegram users like “<em>Hey, thanks to these alerts, I booked a slot in Delhi. Can you do something similar for my folks back in Assam who are struggling to find a slot?</em>”. How do I even refuse to do that?</p><p><strong>6. Collaboration</strong></p><blockquote>Teaming up with other programmers for the common cause.</blockquote><p>By this time, there were many other programmers who had started providing alerts in different parts of the country. I had to expand this to tier-2 and tier-3 districts and hence teaming up with other programmers who were focussing on the urban areas made perfect sense for me. Once again, it was twitter to the rescue and I first met Chandra (@BloreVaccine), for Bangalore and then @Suchdeep who was then handling Pune, Mumbai areas. So with these metro cities out of my purview I then focussed on building a backend that could scale for the providing alerts for the rest of the country.</p><p><strong>7. Going out of the way to help people</strong></p><p>By mid-May, the alerts were crazy popular throughout India. People were still struggling to get the slots but the alerts gave them a decent chance to get them. While we sent alerts to the channels, we tried to understand the usefulness of these alerts to people and help them stay motivated. We conducted polls in the channels to check how many people were able to book the slots based on the recent alerts. We tweeted out tips that can improve their chances of booking slots. We even helped individuals who were struggling to book slots.</p><blockquote>Helping Ramya get a vaccination slot in Ernakulam</blockquote><p>One such incident that I will never forget is the story of Ramya (name changed). Ramya was working in a company based out of Ernakulam where they were asked to join back to work after taking dose 1 vaccination. Vaccination slots for 18+ in Ernakulam was so difficult to get at that time. Because of this, she wasn’t getting a vaccine slot and her company put her on loss of pay during this period! She somehow found my details, created a twitter account and messaged me asking for help. I then quickly booked a slot in Ernakulam using the next alert and then transferred it back to her the next day morning (It was a nerve wracking early morning conference call with @HelpMeHere_Offi volunteers that helped to clinically transfer the slot). Her joy and gratitude she expressed when we transferred the slot to her that morning will forever be in our mind!</p><p>When I look back, May and June of 2021 were two unforgettable months of my life. Of all the code that I have written, this one turned out to have the largest impact!</p><p>And it was all just 400 lines of PHP code!</p><p>Stay safe!<br>Berty Thomas</p><p><em>PS: I will soon write about the </em><a href="https://medium.com/@bertythomas/the-tech-behind-the-under45-in-vaccination-alerts-project-3037a8e31d64"><em>technical implementation of the code</em></a><em> that helped to scale so widely across India overcoming the API limits.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8abe2621c17e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>