Thanks to visit codestin.com
Credit goes to Github.com

Skip to content

Introduce NativeAllocationTracer for tracking native memory allocation sites#816

Merged
saudet merged 7 commits intobytedeco:masterfrom
devjeonghwan:native-memory-tracer
Aug 3, 2025
Merged

Introduce NativeAllocationTracer for tracking native memory allocation sites#816
saudet merged 7 commits intobytedeco:masterfrom
devjeonghwan:native-memory-tracer

Conversation

@devjeonghwan
Copy link
Member

Related to #815

Example

    @Test
    public void testTracer() {
        FloatPointer fp = new FloatPointer(128);

        System.out.println("=======================");
        for (NativeAllocationTracer.Site site : NativeAllocationTracer.getSites()) {
            System.out.println(site.toString());
        }

        fp = null;
        for (int i= 0; i < 128; i++) {
            System.gc();
            Thread.yield();
        }

        System.out.println("=======================");
        for (NativeAllocationTracer.Site site : NativeAllocationTracer.getSites()) {
            System.out.println(site.toString());
        }

        fp = new FloatPointer(128);

        System.out.println("=======================");
        for (NativeAllocationTracer.Site site : NativeAllocationTracer.getSites()) {
            System.out.println(site.toString());
        }

        fp.deallocate();

        System.out.println("=======================");
        for (NativeAllocationTracer.Site site : NativeAllocationTracer.getSites()) {
            System.out.println(site.toString());
        }
    }

Output

=======================
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:18),totalCounts=1,totalBytes=512,liveCounts=1,liveBytes=512,collectedCounts=0,collectedBytes=0]
=======================
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:18),totalCounts=1,totalBytes=512,liveCounts=0,liveBytes=0,collectedCounts=1,collectedBytes=512]
=======================
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:36),totalCounts=1,totalBytes=512,liveCounts=1,liveBytes=512,collectedCounts=0,collectedBytes=0]
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:18),totalCounts=1,totalBytes=512,liveCounts=0,liveBytes=0,collectedCounts=1,collectedBytes=512]
=======================
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:36),totalCounts=1,totalBytes=512,liveCounts=0,liveBytes=0,collectedCounts=0,collectedBytes=0]
Site[location=org.bytedeco.javacpp.TracerTest.testTracer(TracerTest.java:18),totalCounts=1,totalBytes=512,liveCounts=0,liveBytes=0,collectedCounts=1,collectedBytes=512]

@saudet
Copy link
Member

saudet commented Jul 30, 2025

Looking good, but let's move the NativeAllocationTracer class to the tools package, and let's access it dynamically with Class.forName just like PointerBufferPoolMXBean.

@devjeonghwan
Copy link
Member Author

I like the idea too! I was actually considering moving it to the tools package as well. However, this creates accessibility issues for all the mark/record methods.

The problem with your suggestion is that we'd need to handle 5 methods. Even if we get all the Method objects via reflection during initial time, writing all that verbose reflection code at every call site seems weird.

Making those methods to public would solve the accessibility issue, but then we got pollution problem from external calls, which isn't an easy decision either...

I'm curious is there a compelling reason why it must be moved to tools? What specific benefits would we gain that outweigh these complications?

@saudet
Copy link
Member

saudet commented Jul 31, 2025

Putting everything we don't need at runtime in a different package simply avoids namespace pollution, that's all.

And if we don't link dynamically, then the JVM needs to be able to load that class even when it's not needed.

@saudet
Copy link
Member

saudet commented Jul 31, 2025

I suppose we could leave it as package private, that also prevents namespace pollution, but we still need a way to load it dynamically. In the case when we don't need to add "import" statements, does the JVM still loads all classes that might not even get used?

@saudet
Copy link
Member

saudet commented Jul 31, 2025

Huh, I guess not. With Foo.java and Bar.java like this

public class Foo {
    public static void main(String[] args) {
        System.out.println("foo");
        if (args.length > 0)
            Bar.bar();
    }
}
class Bar {
    static void bar() { System.out.println("bar"); }
}
$ javac Foo.java Bar.java
$ java Foo bar
foo
bar
$ rm Bar.class
$ java Foo bar
foo
Exception in thread "main" java.lang.NoClassDefFoundError: Bar
	at Foo.main(Foo.java:6)
Caused by: java.lang.ClassNotFoundException: Bar
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	... 1 more
$ java Foo
foo

So I guess having a package private class that doesn't need any imports is fine after all

@saudet
Copy link
Member

saudet commented Jul 31, 2025

However, it's not possible to tell AOT tools like GraalVM to keep those debug classes out of the image. When we use Class.forName, that becomes possible. I guess that's the reason we should use dynamic loading explicitly

@devjeonghwan
Copy link
Member Author

As you suggested, I've modified it to use dynamic references via reflection instead of direct references.

@devjeonghwan devjeonghwan marked this pull request as draft July 31, 2025 10:32
…logging, and synchronized methods for thread safety
@devjeonghwan devjeonghwan marked this pull request as ready for review July 31, 2025 16:31
@devjeonghwan
Copy link
Member Author

What's the status on this PR? Anything else I should work on?

@saudet saudet merged commit 5be669e into bytedeco:master Aug 3, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants