Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Get only needed stats to greatly reduce memory consumption#47

Merged
webdeveric merged 1 commit into
webdeveric:masterfrom
maximeg:fix-memory-allocation-error
Feb 12, 2019
Merged

Get only needed stats to greatly reduce memory consumption#47
webdeveric merged 1 commit into
webdeveric:masterfrom
maximeg:fix-memory-allocation-error

Conversation

@maximeg

@maximeg maximeg commented Feb 6, 2019

Copy link
Copy Markdown
Contributor

I stumbled into an allocation error when trying this manifest plugin:

 95% emitting WebpackAssetsManifest                                                                                     
<--- Last few GCs --->

[79715:0x104002800]   310074 ms: Scavenge 3481.1 (4172.2) -> 3480.3 (4183.7) MB, 14.3 / 0.0 ms  (average mu = 0.300, current mu = 0.380) allocation failure 
[79715:0x104002800]   310425 ms: Mark-sweep 3491.2 (4183.7) -> 3490.2 (4168.7) MB, 315.3 / 0.0 ms  (average mu = 0.256, current mu = 0.168) allocation failure scavenge might not succeed
[79715:0x104002800]   310454 ms: Scavenge 3495.1 (4168.7) -> 3494.4 (4179.7) MB, 14.0 / 0.0 ms  (average mu = 0.256, current mu = 0.168) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0xcb64b24fc7d]
Security context: 0x12e43151d949 <JSObject>
    1: DoJoin(aka DoJoin) [0x12e431505de9] [native array.js:~89] [pc=0xcb64c7cb622](this=0x12e40ed025b1 <undefined>,0x12e42b2726b9 <JSArray[3171]>,3171,0x12e40ed02741 <true>,0x12e40ed09109 <String[1]:  >,0x12e40ed02801 <false>,0x12e40ed025b1 <undefined>,0x12e40ed025b1 <undefined>)
    2: Join(aka Join) [0x12e431505e39] [native array.js:~115] [pc=0xcb64b5e0907...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x100060c8f node::Abort() [/usr/local/Cellar/node/11.8.0/bin/node]
 2: 0x10006134f node::OnFatalError(char const*, char const*) [/usr/local/Cellar/node/11.8.0/bin/node]
 3: 0x1001749ff v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/11.8.0/bin/node]
 4: 0x1001749a0 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/11.8.0/bin/node]
 5: 0x1004300bc v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/Cellar/node/11.8.0/bin/node]

Raising limits with NODE_OPTIONS="--max-old-space-size=4096" was not enough.

Following what one of my team mate did on the plugin we currently use to actually fix the same memory error we had with it (see https://github.com/danethurber/webpack-manifest-plugin/pull/174/files) here is a PR that do the same here (and better with the catchall all option) should fix the issue.

Cheerz.

@maximeg maximeg requested a review from webdeveric as a code owner February 6, 2019 11:31
@maximeg

maximeg commented Feb 6, 2019

Copy link
Copy Markdown
Contributor Author

@codecov-io

codecov-io commented Feb 6, 2019

Copy link
Copy Markdown

Codecov Report

Merging #47 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master    #47   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           2      2           
  Lines         240    240           
  Branches       49     49           
=====================================
  Hits          240    240
Impacted Files Coverage Δ
src/WebpackAssetsManifest.js 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7caabea...8bbadae. Read the comment docs.

@maximeg maximeg changed the title Get only needed stats to reduce greatly memory consumption Get only needed stats to greatly reduce memory consumption Feb 6, 2019
@webdeveric webdeveric self-assigned this Feb 7, 2019
@webdeveric webdeveric merged commit 051243e into webdeveric:master Feb 12, 2019
@rmacklin

Copy link
Copy Markdown

@webdeveric Would you be able to publish a new release with this change?

@webdeveric

Copy link
Copy Markdown
Owner

@rmacklin Its on my todo list. I have some other updates I'm making and I should be able to publish an update this weekend.

@rmacklin

Copy link
Copy Markdown

Awesome, thank you!

@edwardchen

Copy link
Copy Markdown

Can you publish an update? It would be really helpful if you did!

@chrisvoll

Copy link
Copy Markdown

@webdeveric Could you publish this change? It improves our recompile times pretty dramatically, and I imagine it would help others as well!

@rmacklin

Copy link
Copy Markdown

@webdeveric I noticed some recent commits were merged in #70. Any chance you'd be able to publish a new release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants