When the compilation uses A LOT of files, the ManifestPlugin uses A LOT of memory.
As an example, our compilation has ~2800 files (only assets: images, videos, documents), the ManifestPlugin used between 8 & 10GB of RAM to complete. This is way too much!
I've tracked the issue to be at line:
https://github.com/danethurber/webpack-manifest-plugin/blob/bcca8907df46ad6b57199d34d7be876f95170a52/lib/plugin.js#L58
And more specifically, the toJson() call.
This method is used to get easy-to-use object with information about the webpack compilation.
The plugin only uses the assets field of the generated object, here:
https://github.com/danethurber/webpack-manifest-plugin/blob/bcca8907df46ad6b57199d34d7be876f95170a52/lib/plugin.js#L88
The very high memory usage happens while generating the fields related to modules, an easy fix is to disable this part:
-var stats = compilation.getStats().toJson();
+var stats = compilation.getStats().toJson({
+ modules: false,
+});
It'll still generate other data that the plugin doesn't use.
Another fix is to disable everything we don't need:
-var stats = compilation.getStats().toJson();
+var stats = compilation.getStats().toJson({
+ version: false,
+ hash: false,
+ timings: false,
+ builtAt: false,
+ env: false,
+ publicPath: false,
+ outputPath: false,
+ entrypoints: false,
+ chunks: false,
+ chunkGroups: false,
+ children: false,
+ modules: false, // the memory hungry part is disabled here
+
+ assets: true, // only get assets information
+});
These two diffs resolves the problem, and the compilation uses normal amount of memory.
Let me know what you think about that
When the compilation uses A LOT of files, the
ManifestPluginuses A LOT of memory.As an example, our compilation has ~2800 files (only assets: images, videos, documents), the
ManifestPluginused between 8 & 10GB of RAM to complete. This is way too much!I've tracked the issue to be at line:
https://github.com/danethurber/webpack-manifest-plugin/blob/bcca8907df46ad6b57199d34d7be876f95170a52/lib/plugin.js#L58
And more specifically, the
toJson()call.This method is used to get easy-to-use object with information about the webpack compilation.
The plugin only uses the
assetsfield of the generated object, here:https://github.com/danethurber/webpack-manifest-plugin/blob/bcca8907df46ad6b57199d34d7be876f95170a52/lib/plugin.js#L88
The very high memory usage happens while generating the fields related to
modules, an easy fix is to disable this part:It'll still generate other data that the plugin doesn't use.
Another fix is to disable everything we don't need:
These two diffs resolves the problem, and the compilation uses normal amount of memory.
Let me know what you think about that