OutOfMemoryError While Analyzing 124 GB Heap Dump with ParseHeapDump.sh #135
Description
I'm attempting to analyze a large heap dump file (~124 GB) using the ParseHeapDump.sh script provided by the Eclipse Memory Analyzer Tool (MAT). However, the analysis fails with the following error:
java.lang.OutOfMemoryError: Requested length of new long[2,147,483,640] exceeds limit of 2,147,483,639
This appears to be caused by the JVM’s hard limit on array size (Integer.MAX_VALUE). I've attached a complete stack trace for reference.
Subtask: Parsing /home/ec2-user/heap_dumps/heapdump.hprof
Scanning /home/ec2-user/heap_dumps/heapdump.hprof
[..........java.lang.OutOfMemoryError: Requested length of new long[2,147,483,640] exceeds limit of 2,147,483,639.
Consider enabling object discard, see Window > Preferences > Memory Analyzer > Enable discard
at org.eclipse.mat.parser.index.IndexWriter$Identifier.add(IndexWriter.java:129)
at org.eclipse.mat.hprof.HprofParserHandlerImpl.reportInstance(HprofParserHandlerImpl.java:1160)
at org.eclipse.mat.hprof.HprofParserHandlerImpl.reportInstanceOfPrimitiveArray(HprofParserHandlerImpl.java:119 7)
at org.eclipse.mat.hprof.Pass1Parser.readPrimitiveArrayDump(Pass1Parser.java:860)
at org.eclipse.mat.hprof.Pass1Parser.readDumpSegments(Pass1Parser.java:474)
at org.eclipse.mat.hprof.Pass1Parser.read(Pass1Parser.java:225)
at org.eclipse.mat.hprof.HprofIndexBuilder.fill(HprofIndexBuilder.java:82)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.parse(SnapshotFactoryImpl.java:456)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.openSnapshot(SnapshotFactoryImpl.java:224)
at org.eclipse.mat.snapshot.SnapshotFactory.openSnapshot(SnapshotFactory.java:149)
at org.eclipse.mat.internal.apps.ParseSnapshotApp.parse(ParseSnapshotApp.java:145)
at org.eclipse.mat.internal.apps.ParseSnapshotApp.start(ParseSnapshotApp.java:112)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:208)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:143)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:109)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:439)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:271)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:668)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:605)
at org.eclipse.equinox.launcher.Main.run(Main.java:1481)
at org.eclipse.equinox.launcher.Main.main(Main.java:1454)
Are there any recommended solutions or alternative strategies to analyze such large heap dumps with Eclipse MAT, other than using discard options?
Or is this a fundamental limitation due to the array size constraint in the JVM?
Any guidance or suggestions would be greatly appreciated.
Thank you!