<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shahzeb Ali on Medium]]></title>
        <description><![CDATA[Stories by Shahzeb Ali on Medium]]></description>
        <link>https://medium.com/@shahzeb.dataengineer?source=rss-dc8ec1682692------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*9f_3ydlSMUKOTqULXNOJ7g.jpeg</url>
            <title>Stories by Shahzeb Ali on Medium</title>
            <link>https://medium.com/@shahzeb.dataengineer?source=rss-dc8ec1682692------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 18:40:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shahzeb.dataengineer/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GitHub Sponsors Pays Open Source Maintainers. Here Is How It Works.]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/github-sponsors-pays-open-source-maintainers-here-is-how-it-works-6beb79f6203a?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/6beb79f6203a</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[developer-community]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Sat, 21 Nov 2026 17:08:20 GMT</pubDate>
            <atom:updated>2026-11-21T17:13:44.419Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qXu1cbmr8SJoynis.jpg" /></figure><p><em>Most contributors I work with have never set it up. Some do not know it exists. This is the short version of what it is and how to use it.</em></p><p><a href="https://github.com/sponsors">GitHub Sponsors</a> lets people and companies fund open source maintainers directly on GitHub. Monthly sponsorships, one-time payments, and tiers with benefits if you want them. GitHub says more than $40 million has gone to maintainers through it. Sponsorships from personal accounts carry no GitHub fee, so the full amount reaches the developer. Sponsorships from organizations can carry a fee of up to 6 percent.</p><h3>How it works</h3><p>You set up a Sponsors profile on your GitHub account. You write what you maintain and what the money supports, and you add tiers, for example a small monthly amount, a larger one, and a custom one. Sponsors pick a tier, or send a one-time payment. Payouts go out monthly. The first payout takes about 60 days after your first sponsorship, so do not expect money the same week you set it up.</p><h3>Who qualifies</h3><p>Anyone who contributes to an open source project and lives in one of the 103 supported regions can become a sponsored developer. The full list is on the GitHub Sponsors page, so check whether your region is on it. If it is not, GitHub runs a waitlist you can join. Setting up also needs tax information and two-factor authentication on your account. The old Matching Fund, where GitHub matched donations, is closed to new applications, so do not count on it.</p><h3>What to put in your profile</h3><p>Keep it simple. Say what you maintain, who uses it, and what sponsorship supports: your time on reviews, bug fixes, documentation, keeping the project alive. Tiers work best when they are honest. A small tier can be a thank you. A larger tier can include something real, like early access to a release, a shoutout in the changelog, or a monthly call. Do not promise what you cannot deliver. The profile is a promise, and people remember broken ones.</p><h3>Why so few contributors set it up</h3><p>Nobody tells you. The contribution flow never mentions it. You file issues, open pull requests, get merged, and nobody says there is a funding tab on your profile. Most contributors, especially students and early-career developers, assume open source is unpaid by definition. It is not. The money exists. The knowledge does not.</p><p>There is also an awkward part: asking for money feels strange when you do the work for love. But sponsorship is not a tip jar for nothing. It is payment for work people already depend on. Companies run their products on your code. A sponsorship is the smallest way they can pay for that.</p><h3>If you maintain something, set it up</h3><p>If your work is merged, used, and depended on, set up a profile. It takes an afternoon and costs nothing. Link it from your README and your profile. And if you use open source every day, consider sponsoring one maintainer whose work you depend on. That is the whole point of the program.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6beb79f6203a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Merge That Failed on Rows It Was About to Throw Away]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/the-merge-that-failed-on-rows-it-was-about-to-throw-away-c8ac4f0e549f?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/c8ac4f0e549f</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[delta-lake]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[change-data-capture]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Sat, 21 Nov 2026 16:42:55 GMT</pubDate>
            <atom:updated>2026-11-21T16:45:37.356Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*droBJFwIUwEH4-YY.jpg" /></figure><p><em>A delete marker for a key the table never stored, plus an insert in the same batch, and delta-rs refused the merge. The data was fine. The order of two steps was not.</em></p><p>With change data feed enabled, a merge rejects a source batch that holds both a delete marker for a key the target never stored and an insert, with “1 rows failed validation check”. That is the whole error. Nothing in it tells you which row failed, or why, or that the row in question was about to be thrown away anyway.</p><h3>The setup</h3><p>This <a href="https://github.com/delta-io/delta-rs/issues/5097">is issue #5097</a>, and the fix <a href="https://github.com/delta-io/delta-rs/pull/5098">is PR #5098</a>. I reproduced the reported case on released deltalake 1.6.6 before touching anything, and it failed exactly as reported. The neighboring cases already worked: change data feed off, delete-only batch, insert-only batch. Only the combination failed, which is the first hint that the bug is about ordering, not about the data.</p><h3>What actually happens</h3><p>The merge projects source rows that match no clause, operation SourceDelete, as all-null rows with a null `_change_type`. Then `write_execution_plan_v2` validates the whole projected output, before `write_cdc_plan` splits the batches on `_change_type`. The split drops the null `_change_type` rows from both the data file and the change data feed. But validation already rejected them, on the table’s NOT NULL checks. The rows were never going to be written anywhere. Validation failed them anyway.</p><p>And here is the part that makes it hard to catch: the failure only shows when an insert, update, or delete in the same batch opens the null-file partition in the merge barrier. A batch with only the delete marker commits, because the barrier never releases its rows. So the reporter could run the delete marker alone and it worked, run the insert alone and it worked, and put them together and the merge refused. Three cases, two pass, one fails, and the failing one is the only one that exercises the full path.</p><h3>The fix</h3><p>The fix scopes the table’s validation checks to the rows the CDC split keeps, `_change_type IS NOT NULL`, when the write carries change data. It follows the existing guard for the rewrite predicate in the same function, so it is the same pattern the code already uses, applied to the check that was missing it. Rows the split drops are never written anywhere, so skipping validation for them cannot let a bad row through. The overwrite CDC path is unaffected: its rows all carry a non-null `_change_type`.</p><p>Two files in Rust, one in Python. The core change is 14 lines in the execution plan and 109 lines in the merge operation, most of it the regression test.</p><h3>The review</h3><p>ion-elgreco reviewed it on November 20 and asked one thing: can you add a Python test? I added it, `test_merge_cdf_unmatched_delete_marker_with_insert_5097`, parametrized over streamed and non-streamed execution. It runs the reporter’s scenario through the Python binding and asserts both the table contents and the change feed. The Rust regression test, `test_merge_cdc_unmatched_delete_marker_with_insert`, fails without the fix and passes with it. The merge and write test suites pass, and `cargo fmt` and `cargo clippy` are clean.</p><p>The PR is still open, waiting for the next review round. That is normal. What I take from it is the same lesson as the Iceberg fix from two days ago: in a data engine, the order of steps is the logic. Validation before the split means validating rows that will not exist. Validate what you keep, not what you drop.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c8ac4f0e549f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[‘2.6’ Became 2: The Silent Bug I Fixed in PyIceberg]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/2-6-became-2-the-silent-bug-i-fixed-in-pyiceberg-00b6181502a4?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/00b6181502a4</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[apache-iceberg]]></category>
            <category><![CDATA[data-quality]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 12:19:10 GMT</pubDate>
            <atom:updated>2026-11-19T12:21:34.174Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*IQZaXyTJnpmG232B.jpg" /></figure><p><em>A string literal was truncating fractions instead of refusing them, and a predicate quietly started dropping rows. No error, no warning.</em></p><p>In PyIceberg, `literal(“2.6”).to(IntegerType())` returned `LongLiteral(2)` instead of raising. Read that again. The fraction just disappeared. A bound predicate like `x &lt; “2.6”` on an int column silently became `x &lt; 2`, which drops every row where x is 2. And because the same bound literal feeds partition pruning, whole files could be skipped too. No error anywhere. The query runs, returns fewer rows, and looks correct.</p><h3>Finding it</h3><p>I filed this <a href="https://github.com/apache/iceberg-python/issues/4382">as issue #4382</a> on November 15. I verified it with the expression evaluator first, because a bug report without a reproduction is just an opinion. The reproduction was simple: the literal converts, the predicate changes meaning, the rows go missing.</p><p>What made it clearly a bug, and not a design choice, was the code right next to it. The sibling path, `DecimalLiteral.to(IntegerType())`, already rejects fractional values through an existing helper called `_to_integral`, and there is a test spelling out why: rounding would change the predicate, for example `x &gt; 2.6` becoming `x &gt; 3` drops x = 3. The Java reference implementation has no INTEGER or LONG case in `StringLiteral.to()` at all, and refuses the conversion entirely. The StringLiteral path was simply missed. Two paths that should behave the same, and one of them quietly truncated instead.</p><h3>The fix</h3><p>The fix is <a href="https://github.com/apache/iceberg-python/pull/4383">in PR #4383</a>, and it is small on purpose: reuse `_to_integral` in both StringLiteral paths, IntegerType and LongType. Two files, 33 lines added, 9 removed. A fractional string now raises instead of truncating, the same way the DecimalLiteral path already did.</p><p>There is a new test, `test_fractional_string_to_integral_type_raises`, mirroring the existing test on the DecimalLiteral side. The test matters more than the fix here, because the fix is one helper call and the test is what stops the next person from reintroducing the truncation.</p><h3>The review</h3><p>Fokko, the maintainer, approved it on November 17. His words: “I know that we’re changing behavior here, but since it might lead to unexpected behavior, I think we should do it.” He left one comment around testing, apart from that it looked good to him. He merged it on November 18.</p><p>One small thing happened in between that is worth mentioning. Another contributor asked on the issue whether he could take the fix. The fix was already open in my pull request, and I said so. No argument, no drama. The issue was mine to finish because I had filed it with a reproduction and the fix was already in review.</p><h3>Why this one matters</h3><p>This is my 23rd merged pull request, and my fourth in PyIceberg. But the count is not the point. The point is the shape of the bug. A query engine that silently changes what your predicate means is worse than one that crashes, because a crash stops you and a wrong answer does not. The rows are gone, the files are skipped, and nothing in the output tells you. The worst bugs do not crash. They quietly give you the wrong answer. I wrote that in an earlier article, and this is the same shape again, found the same way: read the code next to the code, and ask why two paths that should agree do not.</p><p>If you work with Iceberg, the takeaway is simple: a string literal with a fraction is now refused when it targets an int or long column, instead of being truncated. That is a behavior change, and it is the right one, because the old behavior was wrong in silence.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=00b6181502a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Review Caught a Bug My Fix Missed]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/the-review-caught-a-bug-my-fix-missed-93070e9a867f?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/93070e9a867f</guid>
            <category><![CDATA[data-replication]]></category>
            <category><![CDATA[filament]]></category>
            <category><![CDATA[jira]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[incremental-sync]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 07:31:55 GMT</pubDate>
            <atom:updated>2026-11-18T09:32:22.834Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y4H3Y3nNaxJpMOj2.jpg" /></figure><p><em>Saloni’s reply was positive, her three suggestions are fixed, CI is green, and the one blocker she named is now cleared too. The connector has merged and shipped.</em></p><p>Two days ago I wrote about opening the Jira connector pull request, <a href="https://github.com/galaxy-io/filament/pull/682">PR #682</a>. The story since then is a better one than the opening, because it is a story about a review working the way a review is supposed to work. It caught something my fix missed, and it caught it before the merge.</p><p>CodeRabbit reviewed the pull request seven minutes after it opened and left three comments. I answered all three on Nov 13. The two Major ones were about watermark precision and Jira timestamp normalization. The timestamp one I could answer with evidence: Jira returned timestamps like 2026–11–13T01:09:44.630+0500, the engine rejected them, and I had added that layout to the timestamp parser with a regression test. The full live run then passed all nine resources, 119 records. The value_date_format point did not apply here, because the Jira issues resource sets overlap_seconds to 300 and epoch_millis keeps millisecond precision. The Minor one was a docs fix, and it became a commit the same day: the docs now require a classic API token, because scoped and service-account tokens fail against the site-specific host.</p><p>Then Saloni reviewed it, <a href="https://github.com/galaxy-io/filament/pull/682#issuecomment-6860789298">and her reply</a> was the kind every contributor hopes for. She called it a really thorough PR and said the live validation is exactly what they want to see. The manifest looks good to her: /search/jql with nextPageToken, ORDER BY updated ASC, the overlap, and commit_on_complete. She also liked the classic versus scoped token catch. She asked Leon and Michael to look at the engine part, since value_date_format is a new grammar option every manifest can use, and she left three smaller suggestions.</p><p>All three are fixed, in commit 8c5ba74. First, fail early: the incremental setup now validates the layout and formats the initial watermark eagerly, so a bad config fails at setup instead of sending raw RFC3339 to the API. Second, a layout check: validation rejects a layout that renders the same value for every watermark, so a typo like YYYY-MM-DD fails at manifest load and at tracker setup. Third, myself is out of the default resources, still selectable. Docs updated to match, tests added for all three, and the test suite passes. CI is green on the latest commit: 18 check runs, 15 success, 3 skipped.</p><h3>The second round: one blocker, and it is real</h3><p>Today, Nov 15, Saloni came back. The fixes look good, she said. And then she named one blocker before this can merge, <a href="https://github.com/galaxy-io/filament/pull/682#issuecomment-6895592412">in her comment</a>. It is the CodeRabbit Jira timestamps comment, and she is right that it is not fully answered yet.</p><p>Here is the bug, precisely. My timestampLayouts fix covers the column values. It does not cover the tracker. The incremental tracker reads fields.updated from the raw record and compares it with the time comparator, which only accepts strict RFC3339. Jira’s 2026–11–15T01:09:44.630+0000 fails that. So the first incremental run would fail on the first issue. The live run passed, but it did not exercise this path the way a real incremental pipeline would, and that is exactly the gap Saloni named.</p><p>What she asked for is specific, and it is the right validation: normalize the cursor to RFC3339 before the tracker sees it, and prove it with a real incremental run, a saved pipeline with — sync-mode incremental, run twice with an issue edit in between. Not a unit test alone. A pipeline, run twice, with a real edit in the middle, the same way a customer would run it.</p><h3>Why this is the good kind of bad news</h3><p>A fix that covers one code path but not the other is the classic shape of a silent bug, and it is the shape I spend most of my time hunting in other people’s code. This time it was in mine, and the review found it. That is not embarrassing. That is the process working. The timestamp layout fix was correct as far as it went. It just did not go far enough, because there are two places a timestamp is read, not one, and I fixed the first and missed the second.</p><p>There is also a smaller lesson in the sequence. The first round of review was about the manifest and the docs. The second round is about the engine, and it is sharper, because the engine is shared. A new grammar option and a new timestamp layout are not connector details. Every manifest can use them, which is why Saloni asked Leon and Michael to look at the engine part, and why the blocker is being held to a real incremental run rather than a unit test.</p><p>Update, Nov 17 night: the blocker is cleared and the connector is merged. The fix normalizes the incremental cursor to RFC3339 in commit 5e68e7d2, and the validation Saloni asked for passed all 13 steps: a saved pipeline, — sync-mode incremental, run twice, with an issue edit in between. She said the validation was exactly what was needed and flagged the PR for engine review. Saloni and Leon then approved it, Leon merged it on Nov 17, and it shipped in Filament v0.46.0, listed first in the release notes. The review caught the bug, the fix covered the second code path too, and that is the process working from start to finish.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93070e9a867f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Trello Returns Actions Newest First. That Broke My First Run.]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/trello-returns-actions-newest-first-that-broke-my-first-run-db0814d3e2b2?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/db0814d3e2b2</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[data-replication]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[change-data-capture]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 14:51:01 GMT</pubDate>
            <atom:updated>2026-11-17T14:52:56.985Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Thirteen resources, a live run on a free account, and one pagination blocker that turned into its own engine pull request.</em></p><p>On November 16 I claimed my second Filament connector. The Jira pull request was already in review, so I asked Saloni <a href="https://github.com/galaxy-io/filament/issues/660">on issue #660</a> whether Trello was still available. It was. Same flow as Jira, she said, straight to a pull request, no separate issue needed. By that evening the Trello pull request was up.</p><h3>The build</h3><p>The Trello connector, <a href="https://github.com/galaxy-io/filament/pull/705">PR #705</a>, covers 13 resources over the Trello REST API version 1. Auth is an API key plus token as query parameters, and the actions resource is incremental on the action date, using the since parameter.</p><p>I validated it live against a free Trello account, and the <a href="https://github.com/Shahzeb-DataEngineer/filament/actions/runs/43932949449">run is public</a>. A full read over the 10 default resources returned 16 records. Then the incremental actions pipeline ran: first run, 3 records. I created a test card on the board and ran it again: 4 records. The 3 actions from the first run were re-read inside the 300 second overlap window, and the 4th was the new card. That is the incremental contract working exactly as designed.</p><h3>The blocker</h3><p>Saloni reviewed it the same evening and called it thorough work. Then she named one blocker. The actions resource uses limit 1000 with pagination none, and Trello returns actions newest first. On a board with more than 1000 actions, the first run only gets the newest thousand, and the older actions never arrive.</p><p>This is the shape of bug I write about most: it does not crash. It quietly gives you the wrong answer. A board with 1001 actions would replicate 1000 of them and report success, and nobody would know the oldest action was missing until something downstream did not add up. The limit was in the manifest, but the fix could not live there, because the engine had no pagination strategy for this case. The existing strategies all assume the API hands you a next-page token or a page number. Trello hands you neither for this endpoint. It hands you records, newest first, and a before parameter that takes you further back.</p><h3>The fix that outgrew the connector</h3><p>The fix became a new engine pagination strategy called last_record. The cursor comes from a field of the last record on the page, for example query.before from the last action id. It keeps walking back until a short page, and it errors on a missing field or a repeated cursor. Never a silent stop. That last part matters, because a pagination strategy that stops quietly on bad data is the same bug in a new shape.</p><p>The change touched the shared engine: the new strategy, a change to the Paginator.Next signature for every strategy, the grammar wiring, and a header case-insensitive check in paginationInjectionCollides. Saloni called the strategy clean, and then asked for the part that makes engine changes slow: since it changes the signature for every strategy, it needs maintainer review. So I split it out of the Trello pull request into its own, <a href="https://github.com/galaxy-io/filament/pull/713">PR #713</a>, so the Trello manifest is not blocked on the engine review.</p><h3>The split, the review, and the rebase</h3><p>Saloni’s reply on the split was the one you want to hear: much easier to review. She also passed on a CodeRabbit point that was valid: the cursor-cycle check should reject any cursor seen earlier in the walk, not only the previous one. A repeated cursor from two pages back is the same infinite loop as one from the page before.</p><p>I closed #713 and replaced it <a href="https://github.com/galaxy-io/filament/pull/716">with PR #716</a>, the same change rebased onto current main with just the last_record commits, plus the cursor-cycle fix and trimmed comments. The cursor check now uses a seen set across all earlier pages, with a test where the cursor sequence A, B, A errors on the repeat. One more thing came out of the review: a unit failure on the branch turned out to be a pre-existing data race in a Slack manifest test, not a regression from last_record. The mock handler appends to shared state from concurrent goroutines, so it fails on its own. I reported it as pre-existing rather than quietly working around it.</p><h3>Same shape as Jira</h3><p>This is the second time in a week the pattern has repeated: build the connector, validate it live, get a review, find a blocker, fix it in the engine, split the engine work out so the connector is not blocked. The Jira pull request, <a href="https://github.com/galaxy-io/filament/pull/682">#682</a>, went through the same shape, and it is still open waiting on its engine review. Trello is now in the same position: Saloni said they will come back to it once the engine pull request is merged.</p><p>That is the work. The connector is the visible part, but the review is where the connector becomes trustworthy. A limit of 1000 with no pagination looks harmless in a manifest until someone runs it against a board that has been alive for five years. The next article will say whether Trello and the last_record strategy merged.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db0814d3e2b2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Review Caught a Bug My Fix Missed]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/the-review-caught-a-bug-my-fix-missed-6c93bef4c54c?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c93bef4c54c</guid>
            <category><![CDATA[jira]]></category>
            <category><![CDATA[incremental-sync]]></category>
            <category><![CDATA[filament]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Sun, 15 Nov 2026 17:38:57 GMT</pubDate>
            <atom:updated>2026-11-15T17:57:49.854Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Saloni’s reply was positive, her three suggestions are fixed, CI is green, and one blocker is left. It is a real one, and it is mine to fix.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*APRMEQkX2_bd9LLA.jpg" /></figure><p>Two days ago I wrote about opening the Jira connector pull request, PR <a href="https://github.com/galaxy-io/filament/pull/682">#682</a>. The story since then is a better one than the opening, because it is a story about a review working the way a review is supposed to work. It caught something my fix missed, and it caught it before the merge.</p><p>CodeRabbit reviewed the pull request seven minutes after it opened and left three comments. I answered all three on Nov 13. The two Major ones were about watermark precision and Jira timestamp normalization. The timestamp one I could answer with evidence: Jira returned timestamps like 2026–11–13T01:09:44.630+0500, the engine rejected them, and I had added that layout to the timestamp parser with a regression test. The full live run then passed all nine resources, 119 records. The value_date_format point did not apply here, because the Jira issues resource sets overlap_seconds to 300 and epoch_millis keeps millisecond precision. The Minor one was a docs fix, and it became a commit the same day: the docs now require a classic API token, because scoped and service-account tokens fail against the site-specific host.</p><p>Then Saloni reviewed it, <a href="https://github.com/galaxy-io/filament/pull/682#issuecomment-6860789298">and her reply</a> was the kind every contributor hopes for. She called it a really thorough PR and said the live validation is exactly what they want to see. The manifest looks good to her: /search/jql with nextPageToken, ORDER BY updated ASC, the overlap, and commit_on_complete. She also liked the classic versus scoped token catch. She asked Leon and Michael to look at the engine part, since value_date_format is a new grammar option every manifest can use, and she left three smaller suggestions.</p><p>All three are fixed, in commit 8c5ba74. First, fail early: the incremental setup now validates the layout and formats the initial watermark eagerly, so a bad config fails at setup instead of sending raw RFC3339 to the API. Second, a layout check: validation rejects a layout that renders the same value for every watermark, so a typo like YYYY-MM-DD fails at manifest load and at tracker setup. Third, myself is out of the default resources, still selectable. Docs updated to match, tests added for all three, and the test suite passes. CI is green on the latest commit: 18 check runs, 15 success, 3 skipped.</p><h3>The second round: one blocker, and it is real</h3><p>Today, Nov 15, Saloni came back. The fixes look good, she said. And then she named one blocker before this can merge, in <a href="https://github.com/galaxy-io/filament/pull/682#issuecomment-6895592412">her comment</a>. It is the CodeRabbit Jira timestamps comment, and she is right that it is not fully answered yet.</p><p>Here is the bug, precisely. My timestampLayouts fix covers the column values. It does not cover the tracker. The incremental tracker reads fields.updated from the raw record and compares it with the time comparator, which only accepts strict RFC3339. Jira’s 2026–11–15T01:09:44.630+0000 fails that. So the first incremental run would fail on the first issue. The live run passed, but it did not exercise this path the way a real incremental pipeline would, and that is exactly the gap Saloni named.</p><p>What she asked for is specific, and it is the right validation: normalize the cursor to RFC3339 before the tracker sees it, and prove it with a real incremental run, a saved pipeline with — sync-mode incremental, run twice with an issue edit in between. Not a unit test alone. A pipeline, run twice, with a real edit in the middle, the same way a customer would run it.</p><h3>Why this is the good kind of bad news</h3><p>A fix that covers one code path but not the other is the classic shape of a silent bug, and it is the shape I spend most of my time hunting in other people’s code. This time it was in mine, and the review found it. That is not embarrassing. That is the process working. The timestamp layout fix was correct as far as it went. It just did not go far enough, because there are two places a timestamp is read, not one, and I fixed the first and missed the second.</p><p>There is also a smaller lesson in the sequence. The first round of review was about the manifest and the docs. The second round is about the engine, and it is sharper, because the engine is shared. A new grammar option and a new timestamp layout are not connector details. Every manifest can use them, which is why Saloni asked Leon and Michael to look at the engine part, and why the blocker is being held to a real incremental run rather than a unit test.</p><p>So the pull request is still open, and the work continues: normalize the cursor, run the pipeline twice, and report back. The next article will say whether the blocker is cleared and the connector is merged.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c93bef4c54c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Built the Jira Connector for Filament. Here Is What Is Inside It.]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/i-built-the-jira-connector-for-filament-here-is-what-is-inside-it-9bdb2dd7184b?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bdb2dd7184b</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[jira-cloud]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 07:51:47 GMT</pubDate>
            <atom:updated>2026-11-13T08:54:35.811Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Nine resources, a live run against a real Jira site, two engine fixes the build forced, and three review comments waiting for answers</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*elAhbq4Zx2SFzrsY.jpg" /></figure><p>On Tuesday night I claimed the Jira connector. On Wednesday morning Saloni assigned it to me. On Wednesday night the pull request was open: PR <a href="https://github.com/galaxy-io/filament/pull/682">#682</a>, 14 files, 468 lines added. This is what is inside it, what the live run proved, and what the review already caught.</p><h3>What the connector covers</h3><p>The connector is a Jira Cloud source, built the way Saloni asked for it: a YAML manifest plus a docs page, following the Asana connector, PR <a href="https://github.com/galaxy-io/filament/pull/657">#657</a>, as the example file set. It covers nine resources, all selected by default: myself, projects, issues, users, issue_types, statuses, priorities, resolutions, and fields.</p><p>Authentication is HTTP Basic with an Atlassian account email and API token, against the customer’s own site address as the base URL. Issue search uses POST /rest/api/3/search/jql with nextPageToken pagination, the endpoint Atlassian moved to, not the older /search endpoint. Issues read incrementally on updated, with a 300 second overlap, and the first run reads from 1970–01–01. The connector stays in preview until the maintainers are happy with a live run, and the docs page is registered in the docs navigation and the connectors overview.</p><p>That is the manifest. But building it forced two changes to the engine itself, and those are the interesting part.</p><h3>The two things the build forced</h3><p>The first is a new option called value_date_format. JQL date fields reject a raw RFC3339 watermark, so the incremental block alone could not express this read. The new option takes a Go time layout, or epoch_millis or epoch_seconds, and reformats a time comparator watermark on the wire only. The stored checkpoint and all comparisons keep the raw RFC3339 value, so nothing about the connector’s state changes, only what gets sent to Jira. Validation rejects value_date_format without the time comparator, and the option is documented in the manifest guide. The Jira connector uses epoch_millis, the same approach Airbyte uses for JQL, because JQL takes an unquoted epoch-milliseconds number for date fields.</p><p>The second is a timestamp parsing fix, and the live run caught it. Jira Cloud emits created and updated with a numeric zone offset and no colon, for example 2026–11–13T01:09:44.630+0500. RFC 3339 parsing rejects that spelling, so the engine rejected those rows. The timestamp parser now accepts it, with a regression test. This is the kind of bug that only shows up when you point the code at a real API, and it is exactly why the brief asked for a live run.</p><h3>The live run</h3><p>The pull request was tested against a personal free Jira Cloud site, not a mock. A connection probe hit GET /rest/api/3/myself, and a full read of all nine resources returned 119 records: myself 1, projects 3, issues 24, users 12, issue_types 13, statuses 8, fields 51, priorities 5, resolutions 2. The nextPageToken walk over issues returned the same 24 issues as the full read, across 13 pages, which is the pagination check that matters. Then the incremental check: an epoch-millis JQL query returned zero rows before an update, only the updated issue, FT-3, after I edited it, and zero rows after the latest watermark. That is the whole incremental story in three numbers, and it is the proof the brief asked for.</p><p>The test suite is clean too: go test, go vet, go build, gofmt, and the YAML formatter all pass, and the docs link check reports no broken links. One check could not run in my environment: the docs validator fails extracting the Mintlify framework with a permissions error before validating anything. I said so in the pull request rather than hiding it, because a reviewer will find it anyway.</p><h3>The review, already</h3><p>CodeRabbit reviewed the pull request seven minutes after it opened and left three comments. Two are Major. The first is about precision loss in the lower-bound check: with commit_on_complete and no overlap, a watermark like 2024–11–06T12:00:00.500Z becomes epoch second 1730894400, and an API can correctly return a row at 12:00:00.250Z for that filter, which the check would then mishandle. The second is about normalizing Jira timestamps before watermark comparisons: for an updated value like 2024–11–06T12:00:00.000+0000, the raw string goes to the watermark tracker before projection, and the tracker’s time parser only understands RFC3339, so extraction fails before the row is emitted. The third is Minor: require an unscoped user token instead of a service-account token, because service-account tokens need the api.atlassian.com gateway and requests to the site-specific host can fail with 401 Unauthorized.</p><p>Copilot could not review it at all: the requester had insufficient remaining quota. So the review so far is CodeRabbit’s three comments, and they are good ones. They are about the new engine code, not the manifest, which is where a mistake would have been expensive. Answering them is the next step, then Saloni’s review, then, if the maintainers are happy, a merge.</p><h3>What this build actually was</h3><p>When I claimed this connector, I described it as a YAML manifest plus a docs page. That was true, and it was also incomplete. The manifest is the visible part. Underneath it, one API’s date format forced a new engine option, and another API’s timestamp spelling forced a parser fix. A connector is not a configuration file. It is a negotiation between what an API actually sends and what an engine assumes, and the live run is where that negotiation happens.</p><p>The pull request is open, the review comments are waiting, and the work continues. The next article will say what changed after the review.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bdb2dd7184b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Free Claude Ultra for Open Source Contributors: Who Actually Qualifies (and My Honest Score)]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/free-claude-ultra-for-open-source-contributors-who-actually-qualifies-and-my-honest-score-7a3b4bedfc8c?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a3b4bedfc8c</guid>
            <category><![CDATA[anthropic-claude]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 08:37:34 GMT</pubDate>
            <atom:updated>2026-11-12T08:40:43.994Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Anthropic offers six months of Claude Ultra to the people who keep open source running. I checked my own numbers against all six routes, and the answer is uncomfortable</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-5fWp2l8yXg1CRTg.jpg" /></figure><p>A link landed in my inbox this week: Anthropic’s Claude for <a href="https://claude.com/contact-sales/claude-for-oss">Open Source programme</a> offers six months of free Claude Ultra to open source developers. Ultra is the $200 a month plan, so six months is $1,200 of value, and it includes Claude Code. My first thought was the same as yours probably is: what is the catch?</p><p>There is one, and it is fair. This is not a coupon code and not a free trial for everyone. It is an application, reviewed by Anthropic, for people with a real open source record. I read the official page and a detailed guide to <a href="https://www.thefoundernation.com/claude-ultra-free-six-months/">the programme</a>, and then I did the thing most people skip: I checked my own numbers against every route before getting excited. Here is what I found, and how you can check yours.</p><h3>What the programme actually gives you</h3><p>Claude for Open Source gives approved applicants six months of complimentary Claude Ultra. Ultra is Anthropic’s highest individual tier: up to ten times the weekly usage of the Pro plan, priority access to selected new features, and access to Claude Code for terminal-based coding work. It also includes monthly API credits, though those credits cannot be used to pay for Claude Code usage, so read that part carefully.</p><p>A few conditions matter. You apply through the official programme page, signing in with GitHub, and Anthropic reviews applications on a rolling basis. Up to 10,000 recipients can be approved unless Anthropic raises the cap. If you are approved, you get an activation link and 90 days to use it. The six months run from activation, not from application. The free subscription does not auto-renew into a paid Ultra plan, but if you already had a paid plan, that plan resumes afterwards unless you cancel it. Approval is never guaranteed, even if you meet a threshold.</p><h3>The six routes, in plain words</h3><p>There are six alternative routes under the maintainer track, and you only need one of them, plus the general requirements. Here they are, without the marketing gloss.</p><p>First, package impact: projects you maintain have 500 or more dependent repositories, or 100 or more dependent packages, in aggregate. Second, downloads: your maintained packages get 200,000 or more combined monthly downloads across registries like npm, PyPI, or crates.io. Third, core contributor: you are a listed committer or maintainer of a recognised foundation or language project, such as CPython, Rust, Node.js, an Apache project, CNCF, Kubernetes, Django, or Rails. Fourth, active contributor: you authored 100 or more pull requests merged into public repositories you do not own in the last 12 months. Fifth, community builder: a repository you maintain had 20 or more unique external contributors with merged pull requests in the last 12 months. Sixth, critical infrastructure: a repository you maintain has an <a href="https://github.com/ossf/criticality_score">OpenSSF criticality score</a> of 0.4 or higher.</p><p>On top of that, every applicant must be an individual, at least 18, with a GitHub account in good standing that is at least two years old, public open source activity in the 90 days before applying, and work on at least one project under an OSI-approved licence.</p><p>If you meet none of the six numbers, there is a second door. Anthropic calls it the Ecosystem Impact Track, and the official page puts it plainly: if you maintain something the ecosystem quietly depends on, apply anyway and tell them about it. It is discretionary, not guaranteed, and you have to make the case with evidence: what the project does, who depends on it, and what your role in it is.</p><h3>My honest score</h3><p>Here is my own check, and I will be exact, because inflating these numbers is the fastest way to get rejected. Anthropic says it can revoke a subscription for false or manipulated information, and buying stars or downloads to qualify is explicitly a way to lose it.</p><p>I have 21 merged pull requests in repositories I do not own. The active contributor route needs 100 in the last 12 months. I am not close. I do not maintain a package with 500 dependent repositories or 200,000 monthly downloads. I contribute to Apache projects, including PyIceberg and DataFusion, but I am a contributor, not a listed committer or maintainer, so the core contributor route does not fit either. I do not maintain a repository with 20 external contributors or an OpenSSF score of 0.4.</p><p>So I meet none of the six numerical routes. What I do have is the impact track, and it is a real case, not a stretch. My contributions go to projects the data ecosystem depends on: delta-rs, Filament, DataFusion, PyIceberg, Soda Core. The work is concrete and checkable on <a href="https://github.com/Shahzeb-DataEngineer">my GitHub profile</a>: silent wrong-results bugs caught before they shipped, a duplicate check that was reporting false duplicates, a timestamp parser that rejected valid timestamps. That is the evidence I would put in the 500-word explanation, along with what the projects do and who depends on them. Whether Anthropic agrees is their decision, and I will say so plainly if I apply: this is a discretionary route, and approval is not promised.</p><h3>How to apply well, if you apply at all</h3><p>If your numbers are better than mine, or your impact case is strong, here is what a good application looks like, based on the programme’s own guidance.</p><p>Apply only through the official programme page. Sign in with the GitHub account that holds your actual contributions. Pick the repository that best shows your role and impact, and make sure it is public with its licence visible. Then write the short explanation, maximum 500 words, in this shape: name your route, introduce the project, explain your role, give measurable evidence, explain the wider impact, and describe exactly how you will use Claude. Specific workflows win: reviewing incoming pull requests, finding bugs, improving test coverage, writing documentation, preparing release notes. Generic lines like “I will use Claude for coding” say nothing. And use your own facts. You can use an AI tool to improve the writing, but the numbers and examples must be yours.</p><p>Two things not to do. Do not submit duplicate applications under different emails or accounts; they may be disregarded, and one person gets one subscription. And do not apply if you have no real open source record and hope the reviewer will not notice. They will notice. The programme exists to support the people who already do the work.</p><h3>Why this is worth sharing at all</h3><p>Six months of Claude Ultra is a serious tool for someone who reviews code, hunts bugs, and writes documentation every day. That is my actual workflow, and it is probably yours too if you contribute to open source. The programme is not charity. It is Anthropic recognising that the people who maintain the software everyone depends on deserve better tools to do it with.</p><p>So check your own numbers the way I checked mine. If you meet a route, apply. If you do not, but your project genuinely matters to the ecosystem, the impact track is there for exactly that case. Make it with evidence, keep it honest, and let Anthropic decide. That is the whole trick, and it is the same trick as open source itself: show the work, and let the work speak.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a3b4bedfc8c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Just Got Assigned the Jira Connector for Filament. Here Is the Plan.]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/i-just-got-assigned-the-jira-connector-for-filament-here-is-the-plan-6f965b4b3913?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f965b4b3913</guid>
            <category><![CDATA[filament]]></category>
            <category><![CDATA[galaxy]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[jql]]></category>
            <category><![CDATA[jira]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 08:35:38 GMT</pubDate>
            <atom:updated>2026-11-12T08:36:44.758Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Claimed on Tuesday night, assigned on Wednesday morning, and the brief is already specific: Jira Cloud only, JQL, and a live test</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*InxqEKX_HuuMtESD.jpg" /></figure><p>Last night, at 23:26, I posted a short comment on Filament’s connector tracking issue, issue <a href="https://github.com/galaxy-io/filament/issues/660">#660</a>. I asked for the Jira source connector. This morning, Saloni’s reply was waiting: Jira is yours.</p><p>That sounds simple. It was not a slow process. It was a race, and I almost missed it.</p><h3>The race to claim one</h3><p>Filament, the data replication tool by Galaxy, opened a tracking issue listing the connectors it wants next. Most SaaS sources on that list are a YAML manifest plus a docs page, no Go code, which makes them a good first contribution. The list had Airtable, Jira, Zendesk, Trello, Typeform, Calendly, and a RabbitMQ sink.</p><p>On Nov 10, a contributor named pratim994 claimed Airtable, and Saloni assigned it to him the same day. The next day he opened a scoped issue for it <a href="https://github.com/galaxy-io/filament/issues/678">#678</a>, with base discovery and table metadata in the first scope. That is how fast this moves. One day after the list went up, the first connector was gone.</p><p>I claimed Jira that same evening. A few hours later, another contributor, KhubaibSattar, claimed the Oracle Database Source, issue <a href="https://github.com/galaxy-io/filament/issues/650">#650</a>. Saloni gave that to him too, with one honest caveat: it is blocked on the ADBC migration, so there is nothing to build yet. She will ping him when the first ADBC connector is up.</p><p>So in two days, three connectors found owners. If you are watching a list like this and waiting for the perfect moment, the moment is already passing.</p><h3>What a Filament connector actually is</h3><p>A common misunderstanding is that building a connector means writing a lot of Go. For these HTTP sources, it does not, at least not at first. The connector is a YAML manifest that describes the source, its tables, its auth, and its pagination, plus a docs page that explains how to set it up. The repo has a guide for it, <a href="https://github.com/galaxy-io/filament/tree/main/.agents/skills/filament-http-connector">the filament-http-connector guide</a>, and a recent example to copy: PR <a href="https://github.com/galaxy-io/filament/pull/657">#657</a>, the Asana connector, which shows the full set of files.</p><p>The pattern is clear from how Airtable is being handled. First, open a scoped issue using the connector request template, so the design discussion has its own place. Then build the manifest and docs, then open a pull request. I will follow the same pattern for Jira, starting with the scoped issue.</p><h3>The brief, point by point</h3><p>Saloni’ <a href="https://github.com/galaxy-io/filament/issues/660#issuecomment-6806434102">s assignment reply</a> is unusually specific, and I am glad it is, because it tells me exactly what good looks like.</p><p>First, Jira Cloud only to start. Data Center and Server can come later. Scoping the first version tightly is the right call, and it is also the call that keeps a first connector shippable.</p><p>Second, basic auth with an Atlassian account email plus an API token, which the manifest grammar already supports. No new auth mechanism to invent.</p><p>Third, every customer has their own site address, something like <a href="https://your-site.atlassian.net,/">https://your-site.atlassian.net,</a> so the connector needs a config field used as the base URL. One manifest, many sites, and the site address is configuration, not code.</p><p>Fourth, and this is the detail that would have bitten me if she had not said it: Atlassian moved issue search to /rest/api/3/search/jql with nextPageToken pagination. The older /search endpoint is not the one to build against. Checking the current docs rather than copying an old example is the whole job here.</p><p>Fifth, incremental sync: issues via JQL on updated. That is the part that makes this a real data pipeline concern rather than a one-time export. You do not re-read the whole of Jira every run. You ask for what changed since the last run, and you keep going.</p><p>Finally, the pull request has to list the resources it covers and how it was tested, ideally with a live run. That last phrase matters. A manifest that has never talked to a real Jira site is a guess. Mine will have talked to one.</p><h3>Why Jira, and why me</h3><p>Jira is where most software teams track their work, which makes it exactly the kind of data that belongs in a warehouse. Issues, their status changes, who worked on what, how long things sat waiting: that is reporting data, and right now teams either export it by hand or do not have it at all. A Filament source connector turns that into a pipeline.</p><p>As for why I picked it, the honest answer is that it sits at the crossing of the two things I have actually done in this repo. I validated the Databricks Zerobus sink, which taught me how Filament thinks about destinations and grants. Then I worked on its HTTP and transform code, including the pagination and decimal comparison fixes. I know the codebase well enough to be useful, and a connector is the next step up from fixing one bug at a time: it is owning a small piece of the product.</p><h3>What happens next</h3><p>The work starts now, in this order. Scoped issue first, for the design discussion. Then the manifest and docs page, following the Asana example. Then a pull request with a live run against a real Jira site, and the list of resources it covers.</p><p>I will write about the build itself once there is something real to show, including what the manifest looks like and what broke along the way. For now, the news is simple: Jira is mine, the brief is clear, and the clock is running.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f965b4b3913" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When the Maintainer Ships Your Fix Himself]]></title>
            <link>https://medium.com/@shahzeb.dataengineer/when-the-maintainer-ships-your-fix-himself-8a842eb0488a?source=rss-dc8ec1682692------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a842eb0488a</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[soda-core]]></category>
            <category><![CDATA[filament]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Shahzeb Ali]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 18:06:08 GMT</pubDate>
            <atom:updated>2026-11-11T18:21:44.276Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*L1PkBbLLV59lwBt3.jpg" /></figure><p>There is a specific feeling in open source that nobody warns you about. You find a real bug, you write a fix, your tests pass, and then the maintainer closes your pull request and merges their own version of the same fix. It happened to me twice in one week. This is what it taught me.</p><h3>The first time: the pagination fix</h3><p>I was <a href="https://github.com/galaxy-io/filament">working in Filament</a>, the data replication tool by Galaxy, on a bug in its HTTP connector. When a paginated API returned numeric cursors above 2⁵³, the connector parsed them into a plain map, and JSON rounding silently changed the number. The next request went out with a corrupted cursor, and the corrupted value got checkpointed. My pull request <a href="https://github.com/galaxy-io/filament/pull/589">#589</a>, decoded the paginator body with a decoder that preserves numbers exactly, and added regression tests.</p><p>Then the maintainer, Leon, opened his own pull request <a href="https://github.com/galaxy-io/filament/pull/602">#602</a>, with the same approach, merged it himself, and closed both my PR and a competing contributor’s PR with the same comment: “Superseded by #602.”</p><p>My fix was right. It just was not the fix that shipped.</p><h3>The second time: the decimal comparison fix</h3><p>The next day, same repo, a different bug. Filament’s transform step compared decimal columns as text, so 10 &gt; 9 evaluated to false. My pull request <a href="https://github.com/galaxy-io/filament/pull/597">#597</a>, refused such comparisons at compile time: better to fail loudly than to certify a wrong answer. Leon closed it as “Superseded by <a href="https://github.com/galaxy-io/filament/pull/605">#605</a>.” His PR took a different road entirely. Instead of refusing the comparison, it made text-carried decimals compare numerically.</p><p>This one stung differently. The first time, at least my approach had won. The second time, my approach lost on merit, or at least on maintainer preference. The issue still got fixed, and users got a better outcome than my refusal would have given them. That is hard to argue with, even when your own code is the one being set aside.</p><h3>The time I got it wrong: the soda fix</h3><p>Then came the third case, and it corrected me. I opened a <a href="https://github.com/sodadata/soda-core">fix in soda-core</a> for a multi-column duplicate check bug: on BigQuery and Redshift, the combined-hash separator rendered as an empty string, so different rows could collide into the same key. My PR <a href="https://github.com/sodadata/soda-core/pull/2967">#2967</a>, was closed by the maintainer, Niels-b, who said the issue was addressed in <a href="https://github.com/sodadata/soda-core/pull/2955">#2955</a>.</p><p>My first instinct was to call it a third takeover. Then I checked the dates. PR #2955 had been open since October 23, six days before I opened mine. It was merged one minute before my PR was closed. My PR was not taken over. It was a duplicate. The fix was already in flight, and my duplicate check had missed the linked PR on the issue.</p><p>That is the part of this story I almost got wrong in public, and it is the reason I am writing this carefully. Not every closed PR is a takeover. Sometimes the timing is just against you.</p><h3>What I actually take away from all three</h3><p>First, a closed PR is not wasted work. To write #589, I had to understand Filament’s pagination internals. To write #597, I had to understand its transform compiler. That knowledge stays with me, and both issues are fixed for users today. The code that shipped does not have my name on it, but the bugs are gone, and I can prove I found them.</p><p>Second, correctness is not the whole game. Approach matters, and so does timing. A fix that arrives after the maintainer has already started their own version will lose, even if it is correct. Checking the issue for linked PRs, not just open or closed status, is now a permanent part of my routine.</p><p>Third, do not get bitter. Maintainers carry the review burden for every contribution, and they own the consequences of what merges. When a maintainer ships their own version, it is usually because it fits their codebase better or because they were already halfway there. Treating that as theft would burn bridges with exactly the people whose projects I want to keep contributing to.</p><p>I have 21 merged pull requests across data engineering projects, and three closed ones that taught me more than some of the merges did. The merged ones built my profile. The closed ones taught me how open source actually works. If you contribute long enough, you will collect both kinds. The trick is to keep the second kind from stopping you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a842eb0488a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>