fix: apply transformHeader only once per header in streaming/chunked parsing - #1130
Merged
pokoli merged 1 commit intoAug 24, 2026
Merged
Conversation
transformHeader was invoked twice per header column: once by the core Parser's returnable(), which transforms and de-duplicates the header row in place, and again by ParserHandle.fillHeaderFields() when copying the header into _fields. The second call additionally operated on the already-transformed value, so a non-idempotent transformHeader corrupted the field names (e.g. "Col_X" became "Col_X_X"). This is a regression: header transformation was correct in 5.4.1 and broke in 5.5.x when the duplicate-header renaming logic (mholt#1058) added a transformHeader call inside returnable() without removing the existing one in fillHeaderFields(). A later fix (mholt#1086) added a headerParsed guard but only stopped returnable() from re-running, leaving the double call between the two functions in place. Since returnable() already BOM-strips, transforms and de-duplicates the header row in place, fillHeaderFields() now just collects the already-processed values. Closes mholt#1083
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1083
When
header: trueis combined with atransformHeadercallback, the callback is currently invoked twice per header column. For non-idempotent transforms (e.g. appending a suffix, mapping display names to keys), this corrupts the resulting field names, and since the duplicate-header renaming logic landed, users also see spurious"Duplicate headers found and renamed"warnings and_1-suffixed columns. Multiple users on #1083 report this as a regression from 5.4.1 to 5.5.x, affecting both streaming/chunked and plain parsing.Root cause
Two code paths each apply the header processing:
Parser'sreturnable()BOM-strips, appliestransformHeader, and de-duplicates the header row in place (added in Refactor header renaming logic to adress #1052, #1007 #1058 for duplicate-header renaming).ParserHandle.fillHeaderFields()'saddHeader()then re-appliesstripBomandtransformHeaderto those already-transformed values when copying them into_fields.#1086 ("Only attempt to parse headers once") added a
headerParsedguard, but that only stopsreturnable()from re-running across chunks — the double application betweenreturnable()andfillHeaderFields()remained, so #1083 stayed reproducible after it was merged.Fix
addHeader()no longer re-appliesstripBom/transformHeader; it simply collects the values thatreturnable()has already processed. Header transformation now runs exactly once per column, matching the 5.4.1 behavior while keeping the duplicate-header renaming from #1058.Tests
transformHeaderis called exactly once per header when parsing in chunks, with a non-idempotent transform (fails before this change: headers come out double-transformed).npm test(lint + node mocha + mocha-headless-chrome), 249 node / 251 browser tests passing.