<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aman Gupta on Medium]]></title>
        <description><![CDATA[Stories by Aman Gupta on Medium]]></description>
        <link>https://medium.com/@amangupta678?source=rss-b522ce599313------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5gS4W0IVHxkYkb98C663lA.jpeg</url>
            <title>Stories by Aman Gupta on Medium</title>
            <link>https://medium.com/@amangupta678?source=rss-b522ce599313------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 15:38:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amangupta678/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[typescript-eslint suggests void. With supabase-js, that drops the request.]]></title>
            <link>https://medium.com/@amangupta678/typescript-eslint-suggests-void-with-supabase-js-that-drops-the-request-dc3916248bb5?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc3916248bb5</guid>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[supabase]]></category>
            <category><![CDATA[eslint]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Tue, 20 Oct 2026 11:13:02 GMT</pubDate>
            <atom:updated>2026-10-20T11:13:02.337Z</atom:updated>
            <content:encoded><![CDATA[<p>For the first 39 sends of a new feature in a production app I run, open tracking just didn’t work. The sends went out fine. The opens never got recorded: 39 sends recorded zero opens. No exception, nothing in the logs, every test green. It came down to one word.</p><p>The call looked roughly like this (names changed):</p><pre>void supabase.rpc(&#39;track_event&#39;, { id });</pre><p>That void was a habit. typescript-eslint&#39;s no-floating-promises complains about promises you don&#39;t await, and its message tells you to await it, handle it, or have it &quot;explicitly marked as ignored with the void operator&quot;. The first suggestion it offers is &quot;Add void operator to ignore.&quot; For an eager promise, like a storage upload or an auth call, that&#39;s fine, because the request is already on its way.</p><p>A supabase-js query builder isn’t like that. It’s a lazy thenable, and nothing gets sent until something calls .then() on it. await does that for you. void doesn&#39;t. So void supabase.rpc(...) builds the request and then throws it away.</p><p>It gets worse. Since typescript-eslint 8.0.0, no-floating-promises doesn&#39;t check thenables by default, so a builder you forgot on its own line isn&#39;t flagged by anything. And if you turn checkThenables on to catch those, the rule flags every builder and suggests void first, which is the one fix guaranteed to make sure the request never happens.</p><p>So I built an ESLint plugin that knows the difference: <a href="https://github.com/guptaaman678/eslint-plugin-supabase">eslint-plugin-supabase</a>. It has five rules, each for a Supabase mistake that’s easy to ship without noticing. Four of them don’t produce an error at all.</p><h3>1. The query that never gets sent</h3><p>supabase/no-unsent-query flags a lazy builder that&#39;s voided or just left sitting there as a statement:</p><pre>void supabase.from(&#39;todos&#39;).insert({ title }); // voided: never sent<br>supabase.from(&#39;todos&#39;).update({ done: true }).eq(&#39;id&#39;, id); // never awaited: never sent</pre><p>You get two suggestions, await (inside an async function) and .then(). There&#39;s no autofix, because both change when the request runs, and that&#39;s your call to make. .then() chains, await, return and passing the builder somewhere else are all fine. Storage downloads are lazy too, so they&#39;re covered.</p><h3>2. The subscription that never fires</h3><pre>supabase<br>  .channel(&#39;room&#39;)<br>  .on(&#39;postgres_changes&#39;, { event: &#39;INSERT&#39;, schema: &#39;public&#39;, table: &#39;messages&#39; }, onMessage)<br>  .subscribe((status) =&gt; console.log(status));</pre><p>If public.messages isn&#39;t in the supabase_realtime publication, this channel happily reaches SUBSCRIBED and then never gets a single event. The only error is a system message your subscribe callback never sees. That same app had a subscription like this sitting on an unpublished table while the table took 1,926 writes. It fired zero times.</p><p>supabase/realtime-table-published replays your migrations and reports the subscription, along with the fix: alter publication supabase_realtime add table public.messages;.</p><h3>3. The read that comes back empty</h3><pre>const { data, error } = await supabase.from(&#39;orders&#39;).select(&#39;*&#39;);<br>// data: [], error: null</pre><p>If public.orders has row level security on and no select policy for the role your client uses, this read gives you an empty result and no error. An update or delete in the same situation quietly changes nothing, also without an error. (Inserts and upserts fail loudly with 42501, so the rule doesn&#39;t bother with them.)</p><p>supabase/rls-readable reads your policies from the migrations and reports the read, plus filtered updates and deletes.</p><h3>4. The channel error nobody hears</h3><pre>supabase.channel(&#39;room&#39;).on(&#39;broadcast&#39;, { event: &#39;ping&#39; }, onPing).subscribe();</pre><p>Call .subscribe() without a status callback and CHANNEL_ERROR and TIMED_OUT go nowhere. supabase/subscribe-status-handler asks you to pass one: .subscribe((status, err) =&gt; { ... }). If you already wrap subscribe in a helper that checks the status, add it to allowWrappers.</p><h3>5. The call the 2026–10–30 change will break</h3><p>This one comes with a date, so it gets its own section further down.</p><h3>Why a linter needs your schema</h3><p>Supabase’s own database linter already catches something close to the empty read (the third one above): lint 0008_rls_enabled_no_policy flags every table with row level security on and no policy. But plenty of those tables are like that on purpose. The app only reads them through a function, and &quot;no policy&quot; is the design.</p><p>You can’t tell a deliberate table from a broken one by looking at the database alone. You can’t tell from the client code alone either: supabase.from(&#39;orders&#39;).select() looks fine until you know what the migrations say about orders. You need both. So the schema rules replay supabase/migrations into a model of the database (tables, grants, row level security, policies, publication membership) and check every client call against it. A table that&#39;s only read through a function never trips rls-readable, because there&#39;s no direct client call to report.</p><p>The plugin also works out which role each client acts as. A client created with a publishable or anon key is anon or authenticated. A client created with the service role key bypasses row level security, so rls-readable leaves it alone. When the plugin can&#39;t trace a client at all, it skips the call and counts the skip. When it can trace the client but can&#39;t tell what kind of key it got (createClient(url, config.key)), it treats it like a publishable client by default (unknownKey: &quot;default&quot;), because in client code that&#39;s almost always what it is. Set unknownKey: &quot;skip&quot; if you&#39;d rather skip those calls.</p><h3>2026–10–30: find the call sites before the change does</h3><p>Supabase has announced that from 2026–10–30, new tables, views and sequences in public stop getting automatic grants for anon, authenticated and service_role on every existing project. A migration that creates a table and forgets the grant will apply cleanly, row level security will look right, and the first request will fail.</p><p>After the change, a client call on a new table that the role has no grant for fails with 42501. <a href="https://github.com/guptaaman678/supabase-grants-lint">supabase-grants-lint</a> catches it at the migration. This plugin catches it at the line of client code that will break. A Realtime subscription on such a table fails silently instead: it connects and never receives any data.</p><p>supabase/table-granted knows which privileges each call needs (an upsert needs insert, update and select; a filtered delete needs delete and select) and reports a call when no role the client can act as has them:</p><pre>authenticated has no insert privilege on `public.orders` in your migrations, so this call fails with<br>42501 (permission denied). Fix in a migration: `grant insert on public.orders to authenticated;`</pre><p>Right now, before the change, the replayed defaults still grant almost everything, so this rule is quiet on most projects. And the replay doesn’t look at the calendar. It only sees the change once your project records it, either with the opt-in migration (the revoke SQL Supabase published) or with grants-lint’s since setting. Without one of those, the rule stays quiet after 2026-10-30 too. Once it&#39;s recorded, every table your next migration creates gets checked at the call site, in your editor, before the request fails in production.</p><h3>What it found in 99 public repos</h3><p>I ran the recommended config over 99 public repositories that have Supabase migrations, TypeScript sources and supabase-js. I pinned the commits and linted them as they are. Out of those 99:</p><ul><li>35 of 99 (35.4%) had at least one finding.</li><li>15 of 99 (15.2%) subscribed with postgres_changes to a table their migrations never add to the publication.</li><li>11 of 99 (11.1%) had a read, update or delete that row level security turns into a silent no-op, by their own migrations.</li><li>8 of 99 (8.1%) had a query that’s never sent, either voided or left floating.</li><li>29 of 99 (29.3%) had a .subscribe() with no status callback.</li><li>2 of 99 (2.0%) already had a call with no privilege behind it, before the change.</li></ul><p>Some honest caveats. The corpus only sees migrations, so a table published or a policy added by hand in the dashboard is invisible to it, and a finding can be right about the migrations and still not happen in that repo’s production database. It’s not a random sample of Supabase apps either. Calls the plugin can’t resolve are skipped, so treat these numbers as lower bounds. I also did a blind hand review of 50 schema findings, drawn at random from a fresh run I hadn’t looked at before. It turned up 2 false positives (4.0%), and one of those came from my own corpus harness and is fixed. I’m not naming any of the repositories, and I won’t.</p><h3>Try it</h3><p>You’ll need a TypeScript project with typed linting:</p><pre>npm install --save-dev eslint-plugin-supabase eslint @typescript-eslint/parser &quot;typescript@&gt;=5.8 &lt;6.1&quot;</pre><pre>// eslint.config.mjs<br>import parser from &#39;@typescript-eslint/parser&#39;;<br>import supabase from &#39;eslint-plugin-supabase&#39;;</pre><pre>export default [<br>  {<br>    files: [&#39;**/*.ts&#39;, &#39;**/*.tsx&#39;],<br>    ignores: [&#39;**/supabase/functions/**&#39;],<br>    languageOptions: {<br>      parser,<br>      parserOptions: { projectService: true, tsconfigRootDir: import.meta.dirname },<br>    },<br>  },<br>  supabase.configs.recommended,<br>];</pre><p>Then run npx eslint .. The ignores line keeps Deno edge functions out of typed linting. The two code rules work on any project. The three schema rules look for supabase/migrations next to your code (and read grants-lint.config.json if you have one). If you created your tables in the dashboard, run supabase db pull once so your migrations actually create them. The plugin doesn&#39;t open a network connection, doesn&#39;t read your .env file and doesn&#39;t send telemetry.</p><p>One more step for the 2026–10–30 change. Once your project is past it (or you opted in early from the dashboard), tell the replay: add the opt-in migration (the revoke SQL Supabase published), or set since in grants-lint.config.json to the version of your last migration before the change. Without one of those, the replay keeps the old automatic grants and table-granted stays quiet on new tables.</p><h3>How I built it</h3><p>The schema rules don’t parse SQL themselves. They use the replay engine from supabase-grants-lint, the migration linter I wrote for the same change, through a new engine export. That way the CLI and the editor agree on what your migrations mean.</p><p>Before writing this, I pointed the plugin at the history of the app where both incidents happened. At the commit before the void fix, no-unsent-query reports the call. At the fix, it goes quiet. The subscription was harder. That app adds tables to the publication inside a do block, which a replay can&#39;t read, so the plugin skips those subscriptions instead of guessing (reportUnresolved names the block and asks for plain alter publication statements). With the block rewritten as plain statements, realtime-table-published reports the unpublished subscription and nothing else.</p><p>Every guard in the rules is mutation-tested, and every rule has a test for each case it leaves alone on purpose.</p><p>It doesn’t check the 1,000-row PostgREST limit. For that, use <a href="https://github.com/anabecalab/rowcap">rowcap</a> by anabecalab, which does it well.</p><h3>Try it on your project</h3><p>Run it once before 2026–10–30. If it finds something, or flags something that isn’t a bug, please <a href="https://github.com/guptaaman678/eslint-plugin-supabase/issues">open an issue</a>. A false positive is a bug, and I want to fix it this week.</p><p>I used AI to help draft and edit this post; the plugin, the tests and the numbers are mine.</p><p>Not affiliated with or endorsed by Supabase.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc3916248bb5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supabase stops auto-granting on Oct 30. Here is what breaks.]]></title>
            <link>https://medium.com/@amangupta678/supabase-stops-auto-granting-on-oct-30-here-is-what-breaks-eefc2f40ae48?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/eefc2f40ae48</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[supabase]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[security]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Mon, 28 Sep 2026 16:35:16 GMT</pubDate>
            <atom:updated>2026-09-28T16:47:56.912Z</atom:updated>
            <content:encoded><![CDATA[<h3>Supabase stops auto-granting on Oct 30. Here is what breaks, and five traps the changelog does not mention.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1kP1ilj9aYYU00s1.gif" /></figure><p>I replayed the migration histories of 100 public Supabase projects on a database without automatic grants. In 98 of them, at least one table ended up unreachable through the Data API for some role. In 92, at least one row level security policy could never apply, because the role it names lacks the privilege the policy’s command needs. Not one of the 100 histories contains the opt-in migration.</p><p>That is not a statement about those projects’ production databases: existing tables keep their grants. It is a statement about what their next table, their next preview branch, or their next fresh environment will look like. Here is why, and what to look for.</p><h3>What changes</h3><p>Supabase has announced that from <strong>2026–10–30</strong>, new tables, views and sequences in the public schema stop getting automatic grants for anon, authenticated and service_role on every existing project (<a href="https://github.com/orgs/supabase/discussions/45329">announcement</a>).</p><p>Until now, a default privilege did the work for you. postgres created a table in public, and the three API roles got access to it immediately. You wrote your row level security policies and forgot that grants existed at all.</p><p>After the change, you need to say it:</p><pre>create table public.todos (<br>  id uuid primary key default gen_random_uuid(),<br>  user_id uuid not null references auth.users (id),<br>  title text not null<br>);</pre><pre>alter table public.todos enable row level security;</pre><pre>create policy &quot;read own todos&quot; on public.todos<br>  for select to authenticated using (auth.uid() = user_id);</pre><pre>-- the new part<br>grant select on public.todos to authenticated;<br>grant select, insert, update, delete on public.todos to service_role;</pre><p>Forget the last two lines and nothing complains. The migration applies cleanly, the policy looks right in the dashboard, and the first request returns:</p><pre>{ &quot;code&quot;: &quot;42501&quot;, &quot;message&quot;: &quot;permission denied for table todos&quot; }</pre><p>Existing tables are not affected. The risk is every table you create from now on, and every environment that is built from your migration history instead of from production.</p><h3>Five traps</h3><p>These are the parts that are easy to miss, even after reading the announcement.</p><h4>1. Replaying your history turns the grants back on</h4><p>If your first migration came from supabase db pull, it probably contains lines like these:</p><pre>alter default privileges for role &quot;postgres&quot; in schema &quot;public&quot;<br>  grant all on tables to &quot;anon&quot;;<br>alter default privileges for role &quot;postgres&quot; in schema &quot;public&quot;<br>  grant all on tables to &quot;authenticated&quot;;<br>alter default privileges for role &quot;postgres&quot; in schema &quot;public&quot;<br>  grant all on tables to &quot;service_role&quot;;</pre><p>pg_dump copies your default privileges as they were when you pulled. Replay that file on a preview branch or a local supabase db reset, and every table your later migrations create gets full access again. Supabase&#39;s own branching docs say so. Production no longer has those defaults, so the table that works in your preview branch fails once it ships.</p><p>The local stack has a second version of the same trap: with the current CLI, new tables are exposed automatically unless you set auto_expose_new_tables = false under [api] in supabase/config.toml. A local test run tells you nothing about grants.</p><p>Of the 100 histories, 2 end with default grants still switched on by their own statements, after replaying every file. The fix is a migration that switches them off again, after the baseline.</p><h4>2. The revoke is narrow</h4><p>Per the SQL Supabase published, the opt-in removes select, insert, update and delete on tables, and usage and select on sequences:</p><pre>alter default privileges for role postgres in schema public<br>  revoke select, insert, update, delete on tables from anon, authenticated, service_role;<br>alter default privileges for role postgres in schema public<br>  revoke usage, select on sequences from anon, authenticated, service_role;</pre><p>The old default was grant all. So truncate, references and trigger (plus maintain on Postgres 17 and later) still go to anon and authenticated on every new table. Row level security does not apply to truncate or references. If you want clients to hold only what you granted, revoke the rest yourself:</p><pre>revoke truncate, references, trigger on public.todos from anon, authenticated;</pre><h4>3. service_role bypasses row level security, not grants</h4><p>A common assumption is that the service role key can do anything. It bypasses policies, but Postgres checks privileges before it looks at policies, and service_role is not a superuser. Your edge functions, cron jobs and admin tools that use the service role key get the same 42501 on a table nobody granted to service_role.</p><pre>grant select, insert, update, delete on public.audit_log to service_role;</pre><p>This was the most common finding in the corpus: 97 of the 100 histories create at least one table service_role cannot read or write without the automatic grants.</p><h4>4. Policies without grants are dead</h4><p>A policy never grants access. It only narrows access a role already has. This policy:</p><pre>create policy &quot;insert own messages&quot; on public.messages<br>  for insert to authenticated with check (auth.uid() = sender_id);</pre><p>does nothing unless authenticated holds insert on public.messages. Without the grant, the request fails before the policy is evaluated. The policy looks like access control, reads like access control, and never runs.</p><p>In 87 of the 100 histories, a table has policies for anon or authenticated while that role holds no data privilege on it at all. Counting every mismatch, including a for update policy on a table the role can only select, 92 have at least one policy that can never apply.</p><h4>5. serial columns need their sequence</h4><pre>create table public.orders (<br>  id bigserial primary key,<br>  user_id uuid not null,<br>  total numeric not null<br>);<br>grant select, insert on public.orders to authenticated;</pre><p>The insert still fails. A serial column&#39;s default calls nextval() on its own sequence, and nextval() needs usage (or update) on that sequence. The announced revoke removes sequence usage too:</p><pre>grant usage on sequence public.orders_id_seq to authenticated;</pre><p>Identity columns (generated always as identity) and uuid keys do not need this: Postgres does not check sequence privileges for identity columns, and gen_random_uuid() uses no sequence at all. This one is rare (1 history in the corpus), but when it happens the error names a sequence nobody remembers creating.</p><h3>How the numbers were measured</h3><p>The corpus is 100 public GitHub repositories with at least three files in supabase/migrations, pushed within the last year, not forks, each pinned to a commit. Median history length: 30 migrations. Each history was replayed twice: once as a database without automatic grants would build it (every migration checked), and once as the project is configured today.</p><p>In total the replays flagged 5,820 distinct tables and views as unreachable for at least one role, a median of 21 per project (for scale, 6,601 relations are in scope at the end of the histories). The parser read 103,326 statements and rejected 0.4% of them; each kind of rejected statement was checked against its source, and all were invalid SQL, syntax Postgres does not have, or not SQL at all. Results are published as aggregates only; no project is named.</p><h3>Check your own project</h3><p>I turned these checks into a linter:</p><pre>npx supabase-grants-lint check</pre><p>Run it from your project root, the folder that contains supabase/. If your migrations live somewhere else, point at them: npx supabase-grants-lint check --dir db/migrations.</p><p>It reads your migrations, replays them in order into a model of who holds which privilege on which table and sequence, and prints the grant that fixes each finding:</p><pre>supabase/migrations/20261002120000_add_todos.sql<br>  1:1   error  GL001  public.todos is created without a grant to service_role: ...<br>               fix   grant select, insert, update, delete on public.todos to service_role;<br>  10:1  error  GL003  Policy &quot;read own todos&quot; on public.todos is for select by authenticated,<br>                      but authenticated holds no select privilege on it: ...<br>               fix   grant select on public.todos to authenticated;</pre><p>npx supabase-grants-lint doctor gives a readiness report: whether an opt-in migration exists, whether replaying the history turns automatic grants back on, and which existing tables a fresh database would not expose.</p><p>To keep it from happening again, run it on every pull request:</p><pre>npx supabase-grants-lint init --since next</pre><p>This writes a config and a GitHub Actions workflow. --since next enforces every migration you add from now on and leaves your history alone, so you do not have to fix a long history before you can turn it on. Findings show up as annotations on the pull request.</p><p>It reads files only. It never connects to your database and sends no telemetry.</p><h3>How it was built</h3><p>This started as a test in a production app with about 60 tables. When I opted that project in early, I wanted proof that every table would still be reachable, so I wrote a test that replayed the whole migration history and compared the result with the grants production actually had. Every trap above is something that test caught.</p><p>The open source version is a rewrite, not a copy. It parses migrations with the real Postgres parser (compiled to WebAssembly), and replays them through a small model of grants, default privileges, sequences and policies. Every rule has failing and passing fixtures, and the rules and the model are mutation-tested, so a check that stops catching its bug fails the build. Before release, it ran over the corpus above, and every false positive I found in a hand-checked sample became a fixture and a fix.</p><h3>Try it before Oct 30</h3><ul><li>Run npx supabase-grants-lint doctor on your project.</li><li>Add the <a href="https://github.com/marketplace/actions/supabase-grants-lint">GitHub Action</a>, so the next migration is checked before it ships.</li><li>If it flags something that works, or misses something that does not, please open an issue with the SQL: false-positive reports are the most useful thing you can send.</li></ul><p>The code is MIT licensed at <a href="https://github.com/guptaaman678/supabase-grants-lint">github.com/guptaaman678/supabase-grants-lint</a>. A star helps other people find it before the deadline.</p><p>Not affiliated with or endorsed by Supabase.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eefc2f40ae48" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How we dealt with consistent high RDS CPU usage]]></title>
            <link>https://medium.com/@amangupta678/how-we-dealt-with-consistent-high-rds-cpu-usage-f202c149f380?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/f202c149f380</guid>
            <category><![CDATA[scheduler]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[aws-rds]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[scalability]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Sat, 22 Oct 2022 03:00:43 GMT</pubDate>
            <atom:updated>2022-10-27T07:08:18.528Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Problem statement</strong></p><blockquote>For an Ad posting platform, we had a cron job that has to apply a value-added feature at the almost precise time. To do so, the frequency of cron we had was 5 minutes which was an acceptable delay for the user. In the APAC region, we were serving three markets which have the highest customer base and also the major number of ever-increasing ad-postings(The number of records in the concerned table was ~20, 50 and 100 million per market). For the APAC region, our RDS instances CPU utilisation started shooting out of the roof clocking 100% consistently, and downgrading our services regularly and frequently.</blockquote><p><strong>Context-setting:</strong> Let me give you a bit more context before moving ahead. We had a lambda getting triggered every 5 minutes. Lambda will hit the API of our microservice <em>n</em> times, n being the number of the markets we are serving. For the APAC region, this was 3. We have different databases for each market but on the same amazon aurora instance with 1 master-1 slave strategy.</p><p><strong>Chronology of the issue we faced:</strong> Without getting into specific details, let&#39;s get straight to the point. At one point, the existing cron job was doing absolutely fine and all was well. Then we start observing spikes in RDS CPU utilisation. For some time, we kind of ignored it as the instances were very less. Along with that, we also observed that the response time for the corresponding API is also degrading. That led us to the thought that there is some poor-performing code or DB query in this API. We did some further RCA and found there were a few records that were returning huge resultset to be picked up in some specific cron cycles. Instances of CPU spikes and alerts of service degradation kept increasing till the point it becomes the topmost critical item for our service. Performance was so degraded that our cron job was taking more than 10 minutes sometime to complete one cycle. And meanwhile, the next cron had been triggered.</p><p><strong>Fixing the issue</strong>: To start with we increased the cron window from 5 minutes to 15 minutes. That gave us immediate relief but that could not last long and soon we had to double our slave instances as well resulting 1 master-2 slaves strategy.</p><p>Though we had taken some immediate action we knew that we need to nail this problem right away. Meanwhile, I along with one more team member (<a href="https://medium.com/u/8681f1b8e88d">Sourabh Garg</a>) had already started going through the written code and analysing each and every query being fired. What is the result set being returned for different time windows for different markets. What is the number of records present in different tables and how we are performing the joins.</p><p>We scratched our heads for a couple of days to understand the complete business logic and the relevance of database queries being fired in that context. After a lot of iterations, we came to the conclusion that we can not improve the business logic further plus does not have much to do on the database side as well. Code had a couple of steps wise decisions to be made before finally figuring out what records it had to update. And point to note here is that all of these steps were user-specific. In one go, you will update all eligible records of a single user only. So, the number of database queries being fired was directly proportional to the {number of eligible users X no of ads he had posted}. And obviously, like any other application, this number will continue to grow as well over time.</p><p>There was a pre-processing part(steps-wise decisions) in the process where we were figuring out what users along with their user_ad_ids are eligible for this time window. This business logic was a bit sophisticated resulting in our real-time updates becoming slow. Also, this pre-precessing was happening in every cycle. That was our devil which we need to take out. So we decided to make all that pre-processing well in advance on a daily basis so that our cron job has nothing else to do apart from updating eligible records.</p><p>Finally, our flow looked somewhat like this: every night at some fixed time, we will do all pre-processing for the next 24 hours. We created a separate cron for this. Result needed to be dumped somewhere for persistence, so we created a new table for the same. This table would contain details like user_id, user_ad_id along with the time when it needs to be updated and a few other required details. Since we didn’t need this data on a permanent basis hence we were consistently deleting old data from this table during this cron run itself. Now, when the feature cron job runs, it would simply make a SELECT from this table where time is within the current 5 min window. The result would have all the user_ad_ids that need to be updated which we could do straight away without any much business logic as we had already figured that out. Here no of queries being fired would be proportional to the no of records in the resultset corresponding to the current time window. Other edge case scenarios or race conditions were handled via application code.</p><p>We first deployed this code with the existing revised frequency of 15 minutes. There was a sudden and significant drop in RDS CPU utilisation. After monitoring, we switched back to the original frequency of 5 minutes. And still no spikes in CPU utilisation. After a few days, we finally moved to the original strategy of 1 master-1 slave. We kept monitoring the situation. No alerts. And finally, we even downgraded our RDS CPU instances reducing our infra cost. Still, our services and RDS performance and health were absolutely fine. Only then, we were able to take a deep breath and marked this issue resolved. Savings of 12,000$ an annum was the bonus on infra cost.</p><p>Systems are meant to be ever-evolving. One which is working today might pose a bottleneck tomorrow. Plus, there is no one solution for every system design problem. It can vary depending on business, domain, scale, demography, user behaviour and a lot of other unknown factors. As software engineers, it is our job to make the best guess and design accordingly. And keep it evolving continuously with the learnings we make every day.</p><p>If you like the article, don’t forget to give it a clap and also follow me for more such articles. Feel free to comment with your learnings or any such encounters that relate to this article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f202c149f380" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Importance of HTTP status codes]]></title>
            <link>https://medium.com/@amangupta678/importance-of-http-status-codes-37d4c401d6f6?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/37d4c401d6f6</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[error-handling]]></category>
            <category><![CDATA[http-status-code]]></category>
            <category><![CDATA[api-development]]></category>
            <category><![CDATA[api-integration]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Mon, 12 Sep 2022 15:12:21 GMT</pubDate>
            <atom:updated>2022-09-12T15:15:05.461Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R6Xr41L2tq2tsxER" /><figcaption>Photo by <a href="https://unsplash.com/@visuals?utm_source=medium&amp;utm_medium=referral">visuals</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Software development is a fascinating practice. A person or a group of persons develop software that only they could use, the same or different organisation(s) could use, a particular set of audiences or maybe anyone in the world could use it. That is why industry-standard practices are introduced from time to time so that there would be a minimum disconnect among different developers. All can adhere to almost similar sets of rules and practices, especially in the Internet industry. These are called <a href="https://en.wikipedia.org/wiki/Request_for_Comments">RFCs</a>.</p><p>One similar RFC is there for <a href="https://www.rfc-editor.org/rfc/rfc9110.html">HTTP standards</a> also which talks about various nuances of it. This RFC itself talks about various HTTP status codes and their meanings. You can go through these <a href="https://www.rfc-editor.org/rfc/rfc9110.html#name-status-codes">here</a>. These status codes are actually the response codes. By looking at the HTTP status/response code only, a client can determine what has actually happened at the server when the client made a request to it with a specific set of parameters, headers, request body, <a href="https://www.rfc-editor.org/rfc/rfc9110.html#name-method-definitions">HTTP method</a> etc.</p><p>To my surprise, I have seen multiple instances here and there, when developers naively send <strong>200 OK</strong> as HTTP response code always even when the server is not able to execute the client request in the desired way. Let me first clarify what sometimes developers meant with the <strong>200 OK</strong> status code and what it actually means.</p><p><strong>Developers&#39; understanding-</strong> If the service is up and running, API is working fine, then the HTTP response code should be <strong>200 OK</strong>.</p><p><strong>Actual meaning-</strong> If the service is up and running, API is working fine and the server is able to execute what the client has asked it to do, then the response code should be <strong>200 OK</strong> otherwise not.</p><p>What a thin line between both. Now, what could be the consequences of this? If ignored at a critical place, it could have massive consequences such as system downtime, data inconsistencies, impact on user experience, revenue loss or anything you could imagine of.</p><p>Some examples of improper handling of HTTP status codes could be:</p><ol><li>The client hits the service with the wrong/restricted/expired credentials, and the service returns with a <strong>200 OK</strong> response instead of a <strong>4xx</strong> response. Behind the scenes, the server might be throwing an exception and from the catch block, is still returning a response with HTTP.OK status. This is in complete violation of HTTP API specs.</li><li>The client hits the service with incorrect request parameters/body. That should result in some kind of invalidation or error at the server level. But still, the server code catches the error and responds with an HTTP.OK status. Doesn’t make sense really, does it?</li><li>There is an API contract between the client and server. One of both make some changes but did not communicate to the other party. Further API calls are bound to fail, but no one would really care since all service calls are returning HTTP.OK status.</li></ol><p>There could be a lot more such scenarios. If you are defeating the fundamental principles, then you can’t really predict when would be D-Day for your service/software.</p><p>There was a classic incident that happened in 2019 on the UberEats platform because of such API defiance with Paytm payments API. Give it a read too <a href="https://www.linkedin.com/posts/gergelyorosz_story-bigtech-uber-activity-6909229360352825344-2OAY/"><strong>here</strong></a><strong> </strong>and focus on point #8 mentioned specifically.</p><p>If you like the article, don’t forget to give it a clap and also follow me for more such articles. Feel free to comment with your learnings or any such encounters that relate to this article.</p><p><strong><em>References</em></strong>:</p><ol><li><a href="https://www.rfc-editor.org/rfc/rfc9110.html">https://www.rfc-editor.org/rfc/rfc9110.html</a></li><li><a href="https://www.linkedin.com/posts/gergelyorosz_story-bigtech-uber-activity-6909229360352825344-2OAY/">https://www.linkedin.com/posts/gergelyorosz_story-bigtech-uber-activity-6909229360352825344-2OAY/</a></li><li><a href="https://en.wikipedia.org/wiki/Request_for_Comments">https://en.wikipedia.org/wiki/Request_for_Comments</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37d4c401d6f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Art of scaling database]]></title>
            <link>https://medium.com/@amangupta678/art-of-scaling-database-b310f6132037?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/b310f6132037</guid>
            <category><![CDATA[partitioning]]></category>
            <category><![CDATA[sharding]]></category>
            <category><![CDATA[database-scaling]]></category>
            <category><![CDATA[consistent-hashing]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Mon, 09 May 2022 01:42:20 GMT</pubDate>
            <atom:updated>2022-05-09T01:42:20.391Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AItDF8bJ12RIFPN_" /><figcaption>Photo by <a href="https://unsplash.com/@jankolar?utm_source=medium&amp;utm_medium=referral">Jan Antonin Kolar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Database is an indisposable part of any system. It is hard to imagine any system that does not have a database as one of its many components. Like other components, databases do need scaling too. Along the time, the customer base and interaction grow every day and so does data that overburdens the database. But unlike web/app servers, scaling up the databases isn’t that simple. Since they persist data, they are not the stateless components hence we need to make sure that we pick the correct database server for making any query.</p><p>There are two major approaches for database scaling:</p><ol><li>Vertical scaling</li><li>Horizontal scaling.</li></ol><h3>Vertical Scaling</h3><p>Vertical scaling is scaling up by adding more power to the existing machine. For example, AWS provides a variety of <a href="https://aws.amazon.com/rds/instance-types/">RDS instances</a> as shown in the attached hyperlink to scale your database vertically. You can choose more powerful machines with more memory and CPU as per your needs. However, there are some serious limitations with the vertical scaling:</p><ol><li>Adding up CPU, RAM is possible up to a limit. No matter how powerful machines you use, at some point in the future, you might need a more powerful machine and hence a single machine might not be enough.</li><li>A single machine always exposes your system to the risk of a Single Point Of Failure.</li><li>The overall cost of vertical scaling is too high. You can get an idea about it from the same attached hyperlink.</li></ol><h3>Horizontal scaling(Data Partitioning/Sharding)</h3><p>Horizontal scaling is adding up more database servers. It is most commonly known as Sharding. Sharding distributes a large database into many smaller ones. Each part is called a shard. After a certain scale point, it is cheaper and more feasible to scale horizontally by adding more machines than to grow it vertically by adding a powerful server. There are many approaches to horizontal scaling.</p><h4><strong>Partitioning Methods</strong></h4><ol><li><strong>Range-based(Horizontal) partitioning</strong>- In this, we store a specific range of data in a specific database server. For example, if we have usernames to store, we might opt for storing users starting with ‘a-e’ as the first character in one server, ‘f-j’ in another server, and so on.<br>The key problem with this approach is that if the value whose value is used for sharding is not chosen carefully, then it might lead to unbalanced shards. As in the previous example, what if most of the users of our application have usernames starting with ‘a’. It might lead to what we call a Hotspot or Celebrity problem.</li><li><strong>Feature-based(Vertical) partitioning</strong>- In this, we store different features in different database servers. For example, if we talk of Twitter we may opt to store user profiles in one server, follower-followee list in one server, tweets in one server, and so on.<br>Vertical partitioning is straightforward to implement and has a low impact on applications. The main problem with this approach is that if our application experiences additional growth, then we might have to further partition a feature-specific database across various servers.</li><li><strong>Directory-based partitioning</strong>- In this, we create a lookup service that knows our current partitioning scheme and abstracts it away from the DB access code. We query our directory server that holds the mapping between the tuple key to its DB server. This loosely coupled approach enables us to perform tasks like adding servers to the DB pool or changing our partitioning scheme without having to impact our application.</li></ol><h4>Partitioning criteria</h4><ol><li><strong>Key/Hash-based partitioning</strong>- In this, we apply a hash function to some key attribute of the entity we are storing which will give us a partition number. For example, if we have <strong><em>n</em></strong> DB servers and our ID is a numeric value, the hash function could be <strong><em>‘ID % n’</em></strong>, which will give us the server number where we can read/write that record.<br>The fundamental problem with this approach is that it effectively fixes the total number of DB servers. Adding new servers means changing the hash function which would require redistribution of data and hence downtime for service. This problem is addressed by using <em>Consistent Hashing.</em></li><li><strong>List partitioning</strong>- In this, each partition is assigned a list of values so whenever we want to insert a new record, we will see which partition contains our key and then store it there. For example, we can decide all usernames starting with <em>‘a’ (most common)</em>, <em>‘u,v’(least common),</em> and a few averagely common first characters will be stored in a single partition.</li><li><strong>Round-robin partitioning</strong>- This is a very simple strategy that ensures uniform data distribution. With ’n’ partitions, the ‘i’ tuple is assigned to partition <em>‘i mod n’</em>.</li></ol><h3>Common challenges of Sharding</h3><p>Sharding is a great technique to scale a database but it is far from the perfect solution. It introduces complexities and new challenges to the system.</p><p><strong>Joins and denormalization</strong>- Performing joins on a database that is running on a single server is straightforward, but once a database is partitioned, it is often not feasible to perform joins that span multiple database shards. A common workaround for this problem is to denormalize the database so that queries that previously required joins can be performed from a single table.</p><p><strong>Referential Integrity</strong>- Trying to enforce data integrity constraints such as foreign keys in a sharded database can be extremely difficult. In such scenarios, we enforce constraints at the application code level.</p><p><strong>Rebalancing</strong>- There could be many reasons we have to change our sharding scheme:</p><ol><li>The data distribution is not uniform,i.e., celebrity problem.</li><li>There is a lot of load on a particular shard,i.e., there are too many requests being handled by a particular shard.</li></ol><p>In such cases, either we have to create more DB shards or have to rebalance existing shards, which means the partitioning scheme changed and all existing data moved to a new location. Doing this without incurring downtime is extremely difficult. Consistent hashing is a commonly used technique to solve this problem.</p><p>Database scaling or partitioning is a very crucial scaling activity. Executing it inefficiently might cost us significant downtime or in some cases might result in loss of data integrity, inconsistent data, or similar worse end results. So it is very important that we give appropriate thought and think of all possible scenarios while taking up this activity.</p><p>If you like the article, don’t forget to give it a clap and also follow me for more such articles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b310f6132037" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Git — The Good, The Bad, and The Ugly]]></title>
            <link>https://medium.com/@amangupta678/git-the-good-the-bad-and-the-ugly-54198ab29024?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/54198ab29024</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[good-bad-and-ugly]]></category>
            <category><![CDATA[rebase]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Mon, 14 Feb 2022 18:09:35 GMT</pubDate>
            <atom:updated>2022-02-15T11:33:58.355Z</atom:updated>
            <content:encoded><![CDATA[<h3>Git — The Good, The Bad, and The Ugly</h3><p>Git is a very powerful tool for modern software development. It has a lot of features to offer. Git is the first choice for version control in the current times. As with any other tool, language, or framework; Git also has mostly good, few bad, and some ugly features. As the saying goes- it is good if we know good parts, better if we are aware of bad, and would be best if we can understand and avoid the ugly parts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fkheAhxqWtU083M8" /><figcaption>Photo by <a href="https://unsplash.com/@iniguez?utm_source=medium&amp;utm_medium=referral">Carlos Alberto Gómez Iñiguez</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This article could not cover the exhaustive list especially for the good and the bad parts, and that is not even the intent. The motive behind writing this article is a few recent encounters where using git has been a consistent headache due to its ugly parts. So I thought to share those learnings via this article.</p><h3>The Good Parts</h3><p>I am sure git would have a lot of good parts. I would like to highlight a few of them.</p><h4>Pull Request</h4><p>Git has this concept of Pull Request which makes git stand apart from its peers. It&#39;s not really git but git-based services that provide this feature. Instead of merging one branch or your changes directly to another branch, you should raise a Pull Request(PR). Pull request highlights the differences or changes that would be merged from the source branch to the destination branch. PR can be then reviewed by other peers and they can either approve and merge the same or comment on PR with the feedback. PR has another use case for contributing to other code repositories where we don’t have the push rights as in the case of open source and all.</p><p><strong>Commit<br></strong>Commit in git vs other VCS(version control systems) like SVN is altogether different though from the outside, it might look the same. <em>Git commit</em> records the changes in your local and does not upload your changes to the remote repository. So you can make as many commits as you wish and then do a single-time <em>git push </em>to upload your changes to remote. Git has totally different concept of tracking the changes and it also provides a bunch of other stuff which you can do before uploading your commits to remote like merge, rebase, squash, etc.</p><p><strong>Meaningful/Patches Commit<br></strong> Well, this might be something unusual but yes git encourages meaningful commits. Meaningful commit means a single commit has only the relevant changes for the task it is intended to be. A relevant example could be let’s say you fixed a bug. During the same, you figured out that code could be refactored and you did refactor. Now, refactoring was not intended for bug fix, it was an exercise for tech debt. Now either you can make the changes themselves as a two-step process- first, make the changes necessary for code fix and commit it followed by another set of changes for refactoring and commit it. Or you can make the changes in one go and do multiple commits of smaller patches. Git has this feature of patched commits and it is good in such cases. Use the following command to do that.</p><blockquote>git add -p &lt;file-name&gt;</blockquote><p><strong>Staging area<br></strong>Staging area is one of git’s unique features. It may be a little challenging to understand it on the first encounter especially if you have earlier experience of SVN. Staging area is one of <a href="https://www.atlassian.com/git/tutorials/undoing-changes/git-reset">three trees of git</a> along with — working directory and commit history. This is because of the staging area only that commit in git is much more advanced and powerful than any other centralized VCS.</p><p>This list is not exhaustive for sure and might go very long but this list has highlighted some frequently used ones.</p><h3>The Bad Parts</h3><p>Bad parts are those which by functionality don’t serve any meaningful purpose. Git has no bad parts as such but if I have to highlight any, then there would be only one.</p><p><strong>Rebase<br></strong>Rebase and merge both have similar purpose- to merge or unify one branch into target branch so that changes in source branch can be reflected into target branch. When we merge, git log shows actual history of work,i.e., where a specific change was made, when it was merged into another branch, how the differenct branches have been evolved over course of project, and all. On the other hand, Rebase is done to keep git history clean. It will be as clean as you have been making changes in the single branch only. It might be simple to understand for the users who want to keep history clean but it actually is corrupting the history. It could be a good part for the smaller projects and less number of parallel developers but for larger parts it would be a tough one.</p><h3>The Ugly Parts</h3><p>These are those features which we start to abuse on our comfort. Intention or purpose was very thoughtful but given the flexibility around those, we have used them to make our life more painful rather than peaceful.</p><p><strong>Master not as master<br></strong>Master branch serves a very specific purpose. Master branch code has some charactersitcs like- it is stable, it is the latest, it is either production ready or resembles the latest deployed code in production.</p><p>So far, I have never experienced master branch not adhering to master specifications till recent. And believe me what a pain it is. It is such a headache that you will regret it for your life. BTW, if you think you have some other branch adhering to above specifics then it is totally fine. Master is just a convention and nothing else but if you have no such branch then you are in deep trouble.</p><p><strong>Rebase<br></strong>Rebase is actually the most ugly part of git for many reasons. One, it has no functional significance. Second, people really doesn’t know how and when to use rebase. Third, for rebase you may have to resolve merge conflicts n number of times if you have git history of n commits.</p><p>There might be few more reasons but I think these are enough to rest my case. Rebase is supposed to perform at your dev branch for local commits before pushing them to remote repository. If you rebase already pushed comits, then you are in trouble for sure. <em>Git merge</em> considers three commits before merging- the last commit of source branch, the last commit of target branch and the common ancestor of both. In rebase, all your local commits are temporarily moved out first, then remote branch is synced and then the local commits are played on top of that one by one. There is a chance that you might get conflicts with every commit. If you don’t, feel yourself very lucky and your team a very good one.</p><p>Every language or framework has good, bad and ugly parts. So it is very critical to understand those throughly as a developer. Getting hang of what works best and what not is as much important as coding or any other activity.</p><p>If you like the article, please don’t forget to give a clap and follow me if it was worth your time and read.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54198ab29024" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GIT basics for beginners]]></title>
            <link>https://medium.com/@amangupta678/git-basics-for-beginners-bab6a0f775e1?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/bab6a0f775e1</guid>
            <category><![CDATA[svn]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[version-control]]></category>
            <category><![CDATA[bitbucket]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Sun, 06 Feb 2022 05:21:05 GMT</pubDate>
            <atom:updated>2022-02-06T05:21:05.087Z</atom:updated>
            <content:encoded><![CDATA[<p>Git has become the modern standard for software development. Git is a free and open-source version control system. Unlike other centralized version control systems such as SVN or CVS, Git is distributed.</p><h4>Need of VCS</h4><p>Why do we need VCS aka Version Control System!! For software projects, where multiple people are working on the same codebase, we need to have some toolkit that can help us maintain multiple versions and efficiently manage code changes from each individual. Also, it should be able to maintain a history of each and every change and basic info about it like — what were the changes done, who made it and when. All such challenges of code management for distributed teams/individuals were addressed by VCS. Git has become much more than a VCS nowadays and finds its place in every stage of software development in one way or another.</p><p>In this article, we will get familiar with the basic terminologies that are used quite frequently and super must to be aware of.</p><h3>Getting familiar with Git Jargons</h3><p><strong>Repository</strong>- A repository is a kind of warehouse for individual projects. It allows you to save versions of your code which you can access at any time.</p><p><strong>Clone</strong>- Git clone refers to making a copy/clone of the target repository on your local machine. To work upon any repository, you need that repository on your local dev machine and for that, you start with git clone.</p><p><strong>Branch</strong>- Unique branches represent individual tasks/features being worked upon by team members in discretion(or you can say in parallel). As the destiny of a river is to merge within sea ultimately, likewise the destiny of any branch is to get merged into the master branch.</p><p><strong>Checkout</strong>- Checkout is used to switch between multiple branches of a repository.</p><p><strong>Commit</strong>- Commit refers to a particular snapshot of your repository. When you want to make changes in any branch, you need to commit those changes. A commit tells you a lot of information- like what were the changes made in this particular commit, who made it, the timestamp when the commit was done, in which branch it was done, and a lot more. Each commit is identified by a unique id which we usually call SHA id, hash id, commit id.</p><p><strong>Push</strong>- When you commit your changes, a snapshot is created on your local dev machine only. To upload those changes to the remote and publish to the other team members, you need to push those commits.</p><p><strong>Pull</strong>- Pull refers to downloading the latest version of your repository. When you take a pull, you download all the remote changes on your local machine since the time when you took a pull last time.</p><p><strong>Cherrypick</strong>- Cherrypick refers to applying some commit from one branch into another. If you have to replicate a commit into some other branch then you can cherrypick it from another branch rather than making the same changes again and creating a new commit.</p><p><strong>Fetch</strong>- Fetch command tells your local git to retrieve the latest meta-data info from the remote without doing any file transfer(unlike git pull). It&#39;s more like just checking to see if there are any changes available.</p><p><strong>Head</strong>- Head is the representation of the last commit in the current checkout branch. When you switch branches with git checkout, the HEAD changes and points to the new branch.</p><p><strong>Master</strong>- Master is the default branch of git conventionally. When we create a new git repository, it contains only single branch called master.</p><p><strong>Merge</strong>- Merge is the process of taking changes from one branch into another. As I stated above, in an ideal world, every branch is destined to be merged into master branch ultimately.</p><p><strong>Origin</strong>- Origin is the reference to remote repository. Whenver you use origin, it implies you want the given command/operation to be executed with reference to remote repository.</p><p><strong>Pull Request</strong>- Git provides a very powerful utility in form of Pull request. Rather than directly merging any branch into the master or any other branch, you raise a Pull Request which highlights the difference between the two branches. Approver of the pull request has to review those differences and then approve and merge the same.</p><p><strong>Stashing</strong>- Sometimes you want to switch the branches but you are in middle of some incomplete task. Stashing allows you to get rid of your local changes without committing it. Git stash basically creates a virtual commit in stack form which we can later pop out for applying same incomplete changes. Stashing works on your local repository only.</p><p><strong>Tag</strong>- Tag marks a specific point in git history. They act as bookmark in git history which can be used anytime to lookup for a specific commit and switch to that.</p><p><strong>Reset</strong>- Reset is used to reset the changes. Depending upon where you want to reset the changes- in working directory, in staged index or the commit history; reset has three forms of invocation- soft, mixed and hard.</p><p><strong>Squash</strong>- Squash is used to unify multiple commits into a single one. You can merge several commits into a single commit before forwarding them to others.</p><p>I have tried to explain most common terms that are used in git context. Git is a vast and very powerful tool in present time. It has a lot to offer then what we have discussed here. To deep dive in git concepts, please refer to the official git documentation.</p><ul><li><a href="https://www.atlassian.com/git/tutorials">Git Tutorials and Training | Atlassian Git Tutorial</a></li><li><a href="https://git-scm.com/docs/gittutorial">Git - gittutorial Documentation</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bab6a0f775e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to write an Immutable class in JAVA]]></title>
            <link>https://medium.com/@amangupta678/how-to-write-an-immutable-class-in-java-fb6a02ca5a2a?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb6a02ca5a2a</guid>
            <category><![CDATA[sticktobasics]]></category>
            <category><![CDATA[howtodoinjava]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[immutability]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Thu, 20 Jun 2019 06:40:30 GMT</pubDate>
            <atom:updated>2019-06-20T06:43:29.996Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tuvy_yo9l3DKIW8_" /><figcaption>Photo by <a href="https://unsplash.com/@ulisesbaga?utm_source=medium&amp;utm_medium=referral">Ulises Baga</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>As we leart in the previous <a href="https://medium.com/@amangupta678/how-to-write-inherently-thread-safe-code-in-java-513c949f36d5">article</a>, Immutable class is inherently thread-safe. Apart from that, Immutable class is less prone to error and is more secure. Immutability signifies that any change in the state of object would result in the creation of a new object so that the state of the previous object remains intact.</p><p>To make a class immutable, we need to follow these rules:</p><ol><li><strong>Make all fields private and final<br></strong>All member fields of the class should be private and final. Private so that they can be accessed by only accessor(or getter) method and not from the outside world. Final so that their values or state could not be changed. Ultimately, this is what we want to achieve through Immutable class.</li><li><strong>Restrict your class to be extended by other classes<br></strong>If a class is extensible, it loses its control of immutability. You cannot trust a child class for maintaining immutability of parent class. That is why we need to restrict it from subclassing. You can do it by making the class final. Having said so, that is not the only way to achieve this.<br>(Shoot me in the comment, and we will discuss more on this point.)</li><li><strong>Don’t provide any mutator method<br></strong>Mutator as the word means to mutate or to change. If there is any mutator method present in our class, our class is prone to state change. So we should be careful and avoid any such method. Setters are basic examples of mutators.</li><li><strong>Use and return defensive copies for any mutable component<br></strong>What if your class has a member of type List. Though you have made your member as a final member, still if someone has got the reference of your list, members or elements present inside the list could be changed. Again, this results in state-change of your class object and we don’t want that. This could be prevented by returning a defensive copy of such members. Similarly, if we are initializing any such field by accepting value from outside, we must initialize it after creating a new defensive copy otherwise change in external value would impact the state of our class object too. Date class is one of the mostly used mutable fields.</li></ol><p>After understanding so much about immutable class, let&#39;s try to design a basic one</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/11b238dd4092889e70889b85ee36029d/href">https://medium.com/media/11b238dd4092889e70889b85ee36029d/href</a></iframe><p>See how we have dealt with Date field to assure immutability of class</p><p>If you like the article, don’t forget to give it a clap and also follow me for more such articles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb6a02ca5a2a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to write inherently thread-safe code in JAVA]]></title>
            <link>https://medium.com/@amangupta678/how-to-write-inherently-thread-safe-code-in-java-513c949f36d5?source=rss-b522ce599313------2</link>
            <guid isPermaLink="false">https://medium.com/p/513c949f36d5</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[howtodoinjava]]></category>
            <category><![CDATA[sticktobasics]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[java]]></category>
            <dc:creator><![CDATA[Aman Gupta]]></dc:creator>
            <pubDate>Mon, 10 Jun 2019 09:03:47 GMT</pubDate>
            <atom:updated>2019-06-13T08:20:55.139Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G7PZOhNy5WLFf9RU" /><figcaption>Photo by <a href="https://unsplash.com/@hjrc33?utm_source=medium&amp;utm_medium=referral">Héctor J. Rivas</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>“Are you comfortable with multi-threading”, asks the interviewer. “Yes, very much”, replies interviewee.</p><p>“What does it take to write thread-safe code”, interviewer enquires further. Interviewee replies,“ Simply use <a href="https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html"><em>synchronized</em></a> and you are good to go”.</p><p>Later on, the candidate could not figure out the reason for his rejection.</p><p>Multithreading or concurrency management is the most abused feature of JAVA language. It is abused because developers exploit it without understanding it completely and deeply. They are unaware of the performance hits that it brings along. Apart from the algorithmic time complexity that we talk about most of the times, time taken by a multithreaded program could be significantly less or worse depending upon how you write your code.</p><p>So how should we write optimized multithreaded code? Well, this article is not about writing a multithreaded code but rather how to avoid writing such code and still make your code works smoothly as you want it to be even in the multithreaded environment.</p><h3>Stateless Class</h3><p>Ever wondered how a web application works where each request is served without any conflict or inter-mixing of data. An application server is nothing but a pool of many threads running at the same time serving that many concurrent requests.</p><p>Those who are aware of <a href="https://spring.io/">Spring</a> framework can easily relate it to how we design our Controller, Service, Repository or other such classes. These classes have no member variables other than the bean classes. What is the rationale behind this? The idea is to make our class <strong>stateless</strong> so that there would be no shared data.</p><p>Wait, shared data but among what? Shared data among multiple cached copies of a particular class instance across multiple threads. Consider a controller class A and suppose there are 10 threads running at a particular moment. Each thread would require an instance of A to serve the request. As we all know from java memory model, thread caches the objects in its own thread stack, hence there would be a1, a2, …., a10 cached copies of a single instance of A across all threads(considering A is singleton). Now, if A has no instance field then we don&#39;t have to worry about the state of A’s any instance and we can keep coding without actually worrying about concurrency issues. The idea works irrespective of A being a singleton or not.</p><p>A simple and basic example of a stateless class will be like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f4b39d866f081569006037b2db664ff7/href">https://medium.com/media/f4b39d866f081569006037b2db664ff7/href</a></iframe><p>You see, this class is stateless because it has no data associated with it. It simply performs addition/subtraction on two arguments and returns the result.</p><h3><strong>Immutable Class</strong></h3><p>Now since we understand that we want to avoid sharing of data across multiple threads, what if a thread wants to update some of the fields of a class. Well, we can handle this by always returning a new instance of the class whenever its state changes. We call such classes as <a href="https://docs.oracle.com/javase/tutorial/essential/concurrency/immutable.html"><strong>Immutable class</strong></a>. Immutable classes are inherently thread safe as we don’t have to worry about the state of data being inconsistent across multiple threads. String class is the best and the most commonly used immutable class in java.</p><p>You can easily go through Oracle docs for more understanding regarding Immutable class and how to create them in a neat manner.</p><h3>Atomic Variables</h3><p>Consider the following implementation of Counter class:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9d66539356c12ca02ee44f458c8b3991/href">https://medium.com/media/9d66539356c12ca02ee44f458c8b3991/href</a></iframe><p>This will work perfectly in a single-threaded application, but when we run the same in multi-threading application, we are bound to get inconsistent results. This is because the increment operation consists of three steps(read, update, write) rather than being completed in one step only. It results in lost updates when multiple threads try to update the value of the counter at the same time.</p><p>Using locks( <a href="https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html">synchronized</a> keyword) in such cases would make results of our application consistent, though performance would take a hit. Because locking involves context switch and acquire/release of locks is a time-consuming process. In such cases, we should go for <a href="https://docs.oracle.com/javase/tutorial/essential/concurrency/atomicvars.html">atomic variables</a> which guarantee the update of variables in a single CPU cycle. Hence, no more inconsistencies among multiple threads. Above example using atomic variable would look like as:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c6b60c4b7074f6ccaa03f32cf3200c02/href">https://medium.com/media/c6b60c4b7074f6ccaa03f32cf3200c02/href</a></iframe><p>So, in this article, we see how could we avoid problems of concurrency without actually indulging in concurrency concepts. Using locks should be our last resort to handle concurrency whenever the above points fail in our rescue.</p><p>If you like the article, don’t forget to give it a clap and also follow me for more such articles.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=513c949f36d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>