Remove the .java extension
The file name must not end with .java in order for the java executable to ignore the shebang line. You may use a different extension, but should preferably have no extension at all, as in the JEP example.
From JEP 330 (emphasis added):
When the launcher reads the source file, if the file is not a Java source file (i.e. it is not a file whose name ends with .java) and if the first line begins with #!, then the contents of that line up to but not including the first newline are ignored when determining the source code to be passed to the compiler. The content of the file that appears after the first line must consist of a valid CompilationUnit as defined by §7.3 in the edition of the Java Language Specification that is appropriate to the version of the platform given in the --source option, if present, or the version of the platform being used to run the program if the --source option is not present.
It doesn't need to end with ".sh" as mentioned in your self-answer; to avoid confusion, it probably shouldn't, because the file is not actually a shell script.
Rationale
Jonathan Gibbons, the author of the JEP, discussed the reasoning behind the requirement in a mailing list post. One of the biggest reasons is that they wanted to avoid making changes to the Java Language Specification to require all implementations to ignore a shebang line at the start of Java source files, which would have been a far more impactful change.
Some relevant excerpts from that message:
Shebang scripts are an executable format defined on some, but not all,
platforms. Creating a shebang script is typically more than just adding
an initial first line to a file; it typically involves a number of steps:
a. Add an initial shebang line to the file
b. Rename the file to a "command-friendly" name
c. Make the file executable
d. Install the file in some standard location
While renaming the file to a command-friendly name is optional, it is
also expected to be common practice. For example, a source file named
HelloWorld.java might be installed as helloworld. And, while the JEP
describes use cases for executing a small single-file program with java HelloWorld.java or executing it as a platform-specific shebang script
with just helloworld, it does not seem like there is a common use case
to execute HelloWorld.java.
[...]
Since Java source files are different artifacts to platform-specific
executable scripts, it makes sense to treat them differently, and since
we do not want to change the Java language to support shebang lines, the
suggestion is to amend the JEP and implementation so that shebang lines
are never stripped from Java source files, i.e. files ending in .java.
This avoids the problem of having the ecosystem of tools handling Java
source files having to deal with arbitrary artifacts like shebang
lines.