[JAVA-4309] NullPointerException creating GraalVM native image Created: 22/Sep/21  Updated: 28/Oct/23  Resolved: 27/Sep/21

Status: Closed
Project: Java Driver
Component/s: Internal
Affects Version/s: None
Fix Version/s: 4.3.3

Type: Improvement Priority: Minor - P4
Reporter: Renārs Kudiņš Assignee: Jeffrey Yemin
Resolution: Fixed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Documentation Changes: Not Needed

 Description   

Method sun.nio.ch.DirectBuffer.cleaner() does not take any parameters in Java 8, code https://github.com/mongodb/mongo-java-driver/blob/master/driver-core/src/main/com/mongodb/internal/connection/tlschannel/util/DirectBufferDeallocator.java#L54 passes null array as a paramter to this method. This leads to NullPointerException when building native image with GraalVM (see below for stacktrace).

Steps to reproduce error:

$git clone https://github.com/retriku/graalvm-scala-mongodb-npe.git
$sbt clean compile 'show graalvm-native-image:packageBin'

[error] Fatal error:com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing com.mongodb.internal.connection.tlschannel.util.DirectBufferDeallocator$Java8Deallocator.<init>() 
[error] Parsing context:
[error]    at com.mongodb.internal.connection.tlschannel.util.DirectBufferDeallocator.<init>(DirectBufferDeallocator.java:114)
[error]    at com.mongodb.internal.connection.tlschannel.DirectBufferAllocator.<init>(DirectBufferAllocator.java:37)
[error]    at com.mongodb.internal.connection.tlschannel.TlsChannel.<clinit>(TlsChannel.java:70)
[error]         at com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:126)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:307)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureTypeFlowCreated(MethodTypeFlow.java:282)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:103)
[error]         at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultSpecialInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:363)
[error]         at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:471)
[error]         at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:540)
[error]         at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:539)
[error]         at com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:188)
[error]         at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:172)
[error]         at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
[error]         at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
[error]         at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
[error]         at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
[error]         at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
[error]         at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
[error] Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: java.lang.NullPointerException
[error]         at parsing com.mongodb.internal.connection.tlschannel.util.DirectBufferDeallocator$Java8Deallocator.<init>(DirectBufferDeallocator.java:54)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2578)
[error]         at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:111)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3439)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3391)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3236)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1122)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1007)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
[error]         at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:81)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:212)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
[error]         at com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:131)
[error]         at com.oracle.svm.hosted.SVMHost.parseBytecode(SVMHost.java:709)
[error]         at com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:605)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:163)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:321)
[error]         at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:293)
[error]         ... 14 more
[error] Caused by: java.lang.NullPointerException
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.lambda$foldInvocationUsingReflection$3(ReflectionPlugins.java:374)
[error]         at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
[error]         at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
[error]         at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
[error]         at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
[error]         at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
[error]         at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
[error]         at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.lambda$foldInvocationUsingReflection$4(ReflectionPlugins.java:374)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.traceConstant(ReflectionPlugins.java:565)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.pushConstant(ReflectionPlugins.java:539)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.foldInvocationUsingReflection(ReflectionPlugins.java:394)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins.access$200(ReflectionPlugins.java:95)
[error]         at com.oracle.svm.hosted.snippets.ReflectionPlugins$5.defaultHandler(ReflectionPlugins.java:335)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.graphbuilderconf.InvocationPlugin.apply(InvocationPlugin.java:105)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.graphbuilderconf.InvocationPlugin.execute(InvocationPlugin.java:170)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.applyInvocationPlugin(BytecodeParser.java:2197)
[error]         at com.oracle.svm.hosted.phases.AnalysisGraphBuilderPhase$AnalysisBytecodeParser.applyInvocationPlugin(AnalysisGraphBuilderPhase.java:86)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.tryInvocationPlugin(BytecodeParser.java:2183)
[error]         at com.oracle.svm.hosted.phases.AnalysisGraphBuilderPhase$AnalysisBytecodeParser.tryInvocationPlugin(AnalysisGraphBuilderPhase.java:67)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.appendInvoke(BytecodeParser.java:1889)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genDynamicInvokeHelper(BytecodeParser.java:1762)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeVirtual(BytecodeParser.java:1712)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeVirtual(BytecodeParser.java:1697)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5371)
[error]         at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3431)
[error]         ... 30 more

 

 



 Comments   
Comment by Jeffrey Yemin [ 27/Sep/21 ]

I went ahead and made the change to work around the GraalVM issue.  We'll release it in the next 4.3 patch as well as 4.4.0.

Comment by Githook User [ 27/Sep/21 ]

Author:

{'name': 'Jeff Yemin', 'email': 'jeff.yemin@mongodb.com', 'username': 'jyemin'}

Message: Removed unnecessary parameter to Class.getMethod

As per https://github.com/marianobarrios/tls-channel/commit/f6702edfbc4f1d482f8c36561d0dde75549b357c.

JAVA-4309
Branch: 4.3.x
https://github.com/mongodb/mongo-java-driver/commit/93ad7b6bbd23f139d7fb3c15cc8d65fc03f91609

Comment by Githook User [ 27/Sep/21 ]

Author:

{'name': 'Jeff Yemin', 'email': 'jeff.yemin@mongodb.com', 'username': 'jyemin'}

Message: Removed unnecessary parameter to Class.getMethod

As per https://github.com/marianobarrios/tls-channel/commit/f6702edfbc4f1d482f8c36561d0dde75549b357c.

JAVA-4309
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/ca81ff6a1ada9c25846ca41bf2adb54e67c7ede9

Comment by Jeffrey Yemin [ 25/Sep/21 ]

I also opened https://github.com/marianobarrios/tls-channel/issues/27, where this code is vendored from.

Comment by Renārs Kudiņš [ 23/Sep/21 ]

@Jeffrey Yemin, yes if null argument is omitted then image is created and running it fails with

public abstract jdk.internal.ref.Cleaner sun.nio.ch.DirectBuffer.cleaner()
Exception in thread "main" java.lang.ClassNotFoundException: sun.misc.Cleaner.
This exception was synthesized during native image building from a call to java.lang.Class.forName(String) with constant arguments.
        at graalvm.JavaApp.main(JavaApp.java:9)

This is expected as I am running Java 11.

 

My setup:

  • OpenJDK 64-Bit Server VM (11.0.12+6-jvmci-21.2-b08) for bsd-amd64 JRE (11.0.12+6-jvmci-21.2-b08), built on Jul 19 2021 17:16:18 by "graal" with clang 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)
  • JDK major version: 11
  • OS: macOS BigSur Version 11.6
  • Architecture: AMD64

 

If I run following code:

Method cleanerAccessor = Class.forName("sun.nio.ch.DirectBuffer").getMethod("cleaner", (Class[])null);
System.out.println(cleanerAccessor);
Method cleanMethod = Class.forName("sun.misc.Cleaner").getMethod("clean");
System.out.println(cleanMethod);

Image is not created.

  

I am not very familiar with GraalVM native image builds (first time trying to create one), but my guess is that this could be related to following from https://www.graalvm.org/reference-manual/native-image/Reflection/

Additionally, a call to Class.getMethod(String, Class[]) will be processed only if the contents of the Class[] argument can be determined with certainty. This last restriction is due to the fact that Java does not have immutable arrays. Therefore, all the changes to the array between the time it is allocated and the time it is passed as an argument need to be tracked. The analysis follows a simple rule: if all the writes to the array happen in linear sections of code, i.e., no control flow splits, then the array is effectively constant for the purpose of analyzing the call.

I have also reported to Oracle GraalVM project https://github.com/oracle/graal/issues/3815

 

Comment by Jeffrey Yemin [ 22/Sep/21 ]

renars.kudins@gmail.com

I'm not able to run sbt clean compile 'show graalvm-native-image:packageBin' and I'm not sure why. If you'd like to move this case along, could you try an experiment for me? See what GraalVm does with just this code:

        Method cleanerAccessor = Class.forName("sun.nio.ch.DirectBuffer").getMethod("cleaner");
        System.out.println(cleanerAccessor);
        Method cleanMethod = Class.forName("sun.misc.Cleaner").getMethod("clean");
        System.out.println(cleanMethod);

Does it create the native image successfully?

Comment by Jeffrey Yemin [ 22/Sep/21 ]

Hi renars.kudins@gmail.com

Very interesting report, and I'm surprised this is the first time this has been reported, as I've seen lots of usage of the Java driver with GraalVM, most notably in the Quarkus project. I'm wondering whether you're using any options that are not enabled by most others. Do you know which of these is triggering the analysis?

      "-H:+AllowVMInspection",
      "--no-fallback",
      "--native-image-info",
      "-H:Name=graalvm-npe",
      "-H:Class=graalvm.Main",
      """-H:ClassInitialization=
        |com.mongodb:run_time,
        |graalvm:run_time""".stripMargin.replaceAll("\n", ""),
      "--no-server",
      "-H:+LogVerbose",
      "-H:+ReportExceptionStackTraces",
      "-H:+ReportUnsupportedElementsAtRuntime",
      "-H:+ReflectionPluginTracing",
      "-R:+PrintPointsToStatistics",
      "-R:+PrintImageObjectTree",
      "-R:+PrintAnalysisStatistics"

It's possible that if you disable one of them, you'll be able to work around this.

In terms of why this is happening: I don't have a lot of familiarity with GraalVM, but I wonder whether this is due to a difference in implementation. In the standard Oracle/Open JDK, it's perfectly fine to pass null as the second parameter to Method.getMethod. The method is defined with a varargs parameter:

public Method getMethod(String name, Class<?>... parameterTypes)

so the null that the driver is passing is just indicating that there are no parameters expected to the method that is being looked up.

In OpenJDK, if you follow the code down it eventually gets here:

   private Method getMethod0(String name, Class<?>[] parameterTypes) {
        PublicMethods.MethodList res = getMethodsRecursive(
            name,
            parameterTypes == null ? EMPTY_CLASS_ARRAY : parameterTypes,
            /* includeStatic */ true);
        return res == null ? null : res.getMostSpecific();
    }

So it's handling the null value that the driver is passing. I wonder if GraalVM is doing something different in its implementation of getMethod that's triggering the NPE.

The other interesting thing is that at runtime this code won't actually be invoked at all, since the call is wrapped in a conditional:

 public DirectBufferDeallocator() {
    if (Util.getJavaMajorVersion() >= 9) {
      deallocator = new Java9Deallocator();
      LOGGER.debug("initialized direct buffer deallocator for java >= 9");
    } else {
      deallocator = new Java8Deallocator();
      LOGGER.debug("initialized direct buffer deallocator for java < 9");
    }
  }

This all said, I don't actually know why the driver code is passing null here at all, rather than just doing:

       cleanerAccessor =  Class.forName("sun.nio.ch.DirectBuffer").getMethod("cleaner"); 

I'll investigate that question a little further, and get back to you. But for now, can you see if you can work around this by changing graalVMNativeImageOptions?

Generated at Thu Feb 08 09:01:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.