chore: inline ndjson parse with updated deps #10054
Conversation
b130c73 to
122d730
Compare
122d730 to
ed61e8c
Compare
There was a problem hiding this comment.
Pull Request Overview
Replaces the ndjson dependency with split2 by inlining the ndjson parsing functionality, reducing bundle size by 24% (from 11,638 to 8,853 lines in worker.js).
- Removes
ndjsondependency and addssplit2dependency - Creates custom
ndjsonParse.tsmodule with only the parse functionality needed - Updates import statements in
streamParser.tsto use the new custom module
Reviewed Changes
Copilot reviewed 5 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-workspace.yaml | Updates catalog dependencies, removing ndjson and adding split2 |
| packages/logger/src/streamParser.ts | Updates import to use custom ndjson parse module |
| packages/logger/src/ndjsonParse.ts | New custom implementation of ndjson parse functionality |
| packages/logger/package.json | Updates package dependencies from ndjson to split2 |
| .changeset/small-gifts-wonder.md | Documents the change for release notes |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
packages/logger/src/ndjsonParse.ts
Outdated
| if (row) return JSON.parse(row) | ||
| } catch (e) { | ||
| if (opts.strict) { | ||
| this.emit('error', new Error(`Could not parse row ${row.slice(0, 50)}...`)) |
There was a problem hiding this comment.
The error message truncates the row at 50 characters but doesn't indicate if truncation occurred. Consider adding a conditional indicator like '...' only when the row is actually longer than 50 characters.
| this.emit('error', new Error(`Could not parse row ${row.slice(0, 50)}...`)) | |
| this.emit('error', new Error(`Could not parse row ${row.length > 50 ? row.slice(0, 50) + '...' : row}`)) |
| } | ||
| } | ||
|
|
||
| return split(parseRow, opts) |
There was a problem hiding this comment.
The opts parameter passed to split() contains strict: true which may not be a valid option for split2. The strict option was used in the parseRow function but split2 may not recognize this option.
| return split(parseRow, opts) | |
| return split(parseRow) |
This removes 24% of the lines in
worker.jsBefore and after:
See https://github.com/ndjson/ndjson.js/blob/master/index.js, the whole dep is trivial
parsewas used here, so 2 other deps were not needed at allsplit2is zero-depIt's heavily used: https://www.npmjs.com/package/split2?activeTab=versions
Though drops Node.js <12: https://github.com/mcollina/split2/releases/tag/v4.0.0 😉