-
Notifications
You must be signed in to change notification settings - Fork 98
feat: avoids Bone constructor overhead for each row #919
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughThe pull request optimizes the binary sync service by introducing a new lightweight repository query method ( Changes
Estimated code review effortπ― 3 (Moderate) | β±οΈ ~25 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touchesβ Failed checks (2 warnings)
β Passed checks (3 passed)
β¨ Finishing touches
π§ͺ Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @fengmk2, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a crucial performance enhancement to the binary synchronization service. By optimizing database interactions to retrieve only essential data for diffing, it drastically reduces the overhead of object instantiation, leading to a more efficient and faster synchronization process, particularly beneficial for large datasets. This directly addresses the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with π and π on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a performance optimization by avoiding the overhead of creating full ORM model instances for every existing binary file during the sync diffing process. It achieves this by adding a new repository method listBinaryNameAndDates that fetches only the necessary columns. The changes look good and the added timing logs are helpful. I've found one critical issue in the test files that must be addressed before merging, and a suggestion to refactor duplicated code for better maintainability.
Codecov Reportβ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #919 +/- ##
==========================================
- Coverage 95.37% 95.26% -0.11%
==========================================
Files 197 197
Lines 22104 22138 +34
Branches 2053 2430 +377
==========================================
+ Hits 21081 21090 +9
- Misses 1023 1048 +25 β View full report in Codecov by Sentry. π New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR optimizes the binary synchronization diff operation by avoiding unnecessary Bone ORM constructor overhead. Instead of fetching full BinaryEntity objects for comparison, it introduces a lightweight listBinaryNameAndDates() method that only retrieves the columns needed for diff detection (id, binaryId, name, date). This directly addresses a documented performance bottleneck identified in profiling reports where listBinaries was consuming significant CPU time during sync operations.
Key changes:
- Adds optimized
listBinaryNameAndDates()repository method for partial column selection - Refactors
diff()method to use lightweight data structures for comparison, only creating full entities when changes are detected - Adds timing instrumentation to measure diff operation performance
- Improves test assertions using
assert.match()with regex patterns instead of string includes - Adds comprehensive test coverage to verify only modified items have their
updatedAtchanged
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| app/repository/BinaryRepository.ts | Adds listBinaryNameAndDates() method that selects only essential columns (id, binaryId, name, date) for diff operations, avoiding full entity instantiation overhead |
| app/core/service/BinarySyncerService.ts | Refactors diff() to use lightweight partial data structures for comparison; adds timing instrumentation; creates Binary entities only when changes detected |
| test/core/service/BinarySyncerService/executeTask.test.ts | Enhances test to verify optimization works correctly by checking only changed items have updated timestamps; improves assertions using assert.match(); contains it.only that must be removed |
Comments suppressed due to low confidence (1)
test/core/service/BinarySyncerService/executeTask.test.ts:22
- This test is marked with
it.only, which will skip all other tests in this file. This should be removed before merging to ensure the full test suite runs in CI/CD.
it('should execute "node" task', async () => {
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
π§Ή Nitpick comments (3)
app/repository/BinaryRepository.ts (1)
58-72: Lightweight projection method looks good; consider extracting a shared type aliasThe new
listBinaryNameAndDates()correctly limits the selected columns and avoids entity construction, which matches the performance goal and repository naming conventions. To reduce duplication and keep service/repository types in sync, you might extract the{ id; binaryId; name; date }shape into a exported type (e.g.BinaryNameAndDateRow) and reuse it here and inBinarySyncerService.diff().app/core/service/BinarySyncerService.ts (2)
207-212: Timing instrumentation insyncDir()is correct and improves observabilityCapturing
startTime, computinguseTime, and logging the diff size along with the adapter class name gives useful performance insight without affecting control flow. You may want to break the long template string into multiple concatenated lines or variables if you need to strictly adhere to the 120-character line limit, but functionally this looks solid.
312-315: Diff logic correctly leverages the lightweight projection and preserves identitySwitching to
listBinaryNameAndDates()and building anexistsMapkeyed bynameis a good optimization that avoids unnecessary entity construction while still providingid/binaryIdfor updates. The date-diff and latest-version revalidation branches both create newBinaryinstances with the existingidandbinaryIdpreserved, which should keep the same row identity while updating metadata and avoiding duplicate records. To reduce duplication and keep types tightly coupled with the repository, consider reusing a shared type alias for{ id; binaryId; name; date }rather than repeating the structural type in both files.Also applies to: 338-355, 363-382
π Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
π Files selected for processing (3)
app/core/service/BinarySyncerService.ts(3 hunks)app/repository/BinaryRepository.ts(1 hunks)test/core/service/BinarySyncerService/executeTask.test.ts(4 hunks)
π§° Additional context used
π Path-based instructions (6)
**/*.{js,ts,tsx,jsx}
π CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{js,ts,tsx,jsx}: Use single quotes in JavaScript/TypeScript code (from Prettier configuration)
Use 2-space indentation (from Prettier configuration)
Maintain 120 character line width maximum (from Prettier configuration)
Use trailing commas in ES5 syntax (from Prettier configuration)
Avoid parentheses in arrow functions when possible (from Prettier configuration)
Maximum of 6 function parameters (from Oxlint configuration)
Warn on console usage (from Oxlint configuration)
Disallow anonymous default exports (from Oxlint configuration)
Use ES modules (import/export) syntax throughout
Files:
test/core/service/BinarySyncerService/executeTask.test.tsapp/repository/BinaryRepository.tsapp/core/service/BinarySyncerService.ts
**/*.{ts,tsx}
π CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use strict TypeScript with comprehensive type definitions - avoidanytypes, use proper typing orunknown
Export types and interfaces for reusability in TypeScript
Files:
test/core/service/BinarySyncerService/executeTask.test.tsapp/repository/BinaryRepository.tsapp/core/service/BinarySyncerService.ts
test/**/*.test.ts
π CodeRabbit inference engine (.github/copilot-instructions.md)
test/**/*.test.ts: Test files must use.test.tssuffix
Tests must use@eggjs/mockfor mocking and testing
Tests must useassertfromnode:assert/strictfor assertions
Test files should be organized intest/directory mirroring source structure
Mock external dependencies usingmock()from@eggjs/mockin tests
Use realistic test data created throughTestUtilhelper methods
Clean up after tests - database is reset between test files
Test both success and failure cases - error paths are equally important
Test files should follow naming pattern: describe('[HTTP_METHOD /api/path] functionName()', ...)
test/**/*.test.ts: Test files must use.test.tssuffix
Use@eggjs/mockfor mocking in test files
Useassertfromnode:assert/strictin test files
Test files must mirror source structure intest/directory and test both success and error cases
Files:
test/core/service/BinarySyncerService/executeTask.test.ts
**/*.ts
π CodeRabbit inference engine (CLAUDE.md)
**/*.ts: Use strict TypeScript with proper typing - avoidanytypes, use proper typing orunknowninstead
Use ES modules withimport/exportsyntax throughout the codebase
Use single quotes (') for strings
Use 2-space indentation
Enforce 120 character line width
Use ES5 trailing commas
Limit functions to a maximum of 6 parameters
Do not use console statements - use logger instead
Files:
test/core/service/BinarySyncerService/executeTask.test.tsapp/repository/BinaryRepository.tsapp/core/service/BinarySyncerService.ts
app/repository/**/*.ts
π CodeRabbit inference engine (.github/copilot-instructions.md)
app/repository/**/*.ts: Repository methods must follow naming convention:findSomething,saveSomething,removeSomething,listSomethings
Repositories must contain CRUD operations and query building without business logic
Repositories must inject dependencies using@Inject()decoratorUse repository method naming convention:
findX(single),saveX(create/update),removeX(delete),listXs(multiple, plural)
Files:
app/repository/BinaryRepository.ts
app/core/service/**/*.ts
π CodeRabbit inference engine (.github/copilot-instructions.md)
app/core/service/**/*.ts: Services must implement core business logic without HTTP concerns or direct database access
Services must use@SingletonProto()for service lifecycle management
Services must orchestrate multiple repositories and entities, managing transactions and events
Files:
app/core/service/BinarySyncerService.ts
π§ Learnings (11)
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Test both success and failure cases - error paths are equally important
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:56.815Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-29T15:42:56.815Z
Learning: Applies to test/**/*.test.ts : Test files must mirror source structure in `test/` directory and test both success and error cases
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:56.815Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-29T15:42:56.815Z
Learning: Applies to test/**/*.test.ts : Use `assert` from `node:assert/strict` in test files
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Tests must use `assert` from `node:assert/strict` for assertions
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Clean up after tests - database is reset between test files
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Use realistic test data created through `TestUtil` helper methods
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:56.815Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-29T15:42:56.815Z
Learning: Applies to test/**/*.test.ts : Test files must use `.test.ts` suffix
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Test files should be organized in `test/` directory mirroring source structure
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to test/**/*.test.ts : Mock external dependencies using `mock()` from `eggjs/mock` in tests
Applied to files:
test/core/service/BinarySyncerService/executeTask.test.ts
π Learning: 2025-11-29T15:42:37.586Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-29T15:42:37.586Z
Learning: Applies to app/repository/**/*.ts : Repository methods must follow naming convention: `findSomething`, `saveSomething`, `removeSomething`, `listSomethings`
Applied to files:
app/repository/BinaryRepository.ts
π Learning: 2025-11-29T15:42:56.815Z
Learnt from: CR
Repo: cnpm/cnpmcore PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-29T15:42:56.815Z
Learning: Applies to app/repository/**/*.ts : Use repository method naming convention: `findX` (single), `saveX` (create/update), `removeX` (delete), `listXs` (multiple, plural)
Applied to files:
app/repository/BinaryRepository.ts
𧬠Code graph analysis (1)
app/core/service/BinarySyncerService.ts (3)
app/common/adapter/binary/PrismaBinary.ts (1)
result(75-134)app/common/LogUtil.ts (1)
isoNow(1-3)app/core/entity/Binary.ts (1)
Binary(16-49)
β° Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (16)
- GitHub Check: Agent
- GitHub Check: CodeQL analysis (javascript-typescript)
- GitHub Check: test on mysql (node@22, shard@0/3, enableJSONBuilder@true)
- GitHub Check: test on mysql (node@24, shard@0/3, enableJSONBuilder@false)
- GitHub Check: test on mysql (node@24, shard@1/3, enableJSONBuilder@false)
- GitHub Check: test on mysql (node@22, shard@0/3, enableJSONBuilder@false)
- GitHub Check: test on mysql (node@24, shard@0/3, enableJSONBuilder@true)
- GitHub Check: test on postgresql (node@22, shard@2/3)
- GitHub Check: test on postgresql (node@24, shard@2/3)
- GitHub Check: test on mysql (node@22, shard@1/3, enableJSONBuilder@true)
- GitHub Check: test on postgresql (node@24, shard@1/3)
- GitHub Check: test on postgresql (node@24, shard@0/3)
- GitHub Check: test on postgresql (node@22, shard@0/3)
- GitHub Check: test on postgresql (node@22, shard@1/3)
- GitHub Check: test-deployment
- GitHub Check: build-and-push-image
|
π This PR is included in version 4.14.0 π The release is available on: Your semantic-release bot π¦π |
Add analysis scripts for CPU profiles, heap dumps, and heap allocation profiles: - analyze-cpuprofile.js: Analyzes V8/xprofiler CPU profiles - analyze-heapdump.js: Analyzes V8 heap snapshots - analyze-heapprofile.js: Analyzes heap allocation profiles - compare-profiles.js: Compares two CPU profiles side-by-side Key findings from production profile analysis: - Leoric ORM Bone constructor is the #1 CPU hotspot (up to 5.59% under high load) - JSON parsing is the #1 memory allocator (20.40%) - Tegg DI container adds significant overhead (9% CPU, 16% memory) - PR #919 addresses Bone constructor overhead π€ Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
Add comprehensive CPU profile analysis tools and reports: - analyze-cpuprofile.mjs: Main analyzer with category breakdown - analyze-active-cpu.mjs: Active CPU analysis excluding idle time - analyze-bone.mjs: Bone constructor overhead analysis - analyze-call-graph.mjs: App to ORM call relationships - analyze-routing.mjs: Request routing hotspot analysis - analyze-app-to-framework.mjs: Framework integration analysis Key findings from v4.14.0 profile: - 76% idle time (I/O bound, expected for registry) - Application code uses only 4.5% of active CPU (efficient) - Leoric ORM: 15.2% (Bone constructor is top hotspot) - Egg/Tegg framework: 27.3% (routing, DI, lifecycle) - deep-equal now uses 0% CPU (fixed in PR #919) π€ Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
close #900
Summary by CodeRabbit
Performance
Improvements
βοΈ Tip: You can customize this high-level summary in your review settings.