Is your feature request related to a problem? Please describe.
This is a follow-up to this discussion in the Quarkus tracker: https://github.com/quarkusio/quarkus/issues/45537#issuecomment-2587715601 .
In the Quarkus project, @zakkak is leading an effort to allow throwing exceptions when GraalVM/Mandrel tries to do reflection calls on elements that were not registered for reflection. This effort leads to all sort of discoveries and in the issue above we discovered that Jackson tries to collect information on java.*
classes, which should probably be avoided: these classes can't be annotated with Jackson classes and they are well known so we could probably hardcode what's needed.
We are having this issue with at least AnnotatedFieldCollector
/AnnotatedMethodCollector
/AnnotatedCreatorCollector
. There might be others.
Examples of this issue with stacktraces:
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.util.Date.getDeclaredFields()
without it being registered for runtime reflection. Add java.util.Date.getDeclaredFields() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredFields(DynamicHub.java:1251)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector._findFields(AnnotatedFieldCollector.java:73)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector._findFields(AnnotatedFieldCollector.java:71)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector.collect(AnnotatedFieldCollector.java:48)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.sql.Date.getDeclaredFields()
without it being registered for runtime reflection. Add java.sql.Date.getDeclaredFields() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredFields(DynamicHub.java:1251)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector._findFields(AnnotatedFieldCollector.java:73)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector.collect(AnnotatedFieldCollector.java:48)
com.fasterxml.jackson.databind.introspect.AnnotatedFieldCollector.collectFields(AnnotatedFieldCollector.java:43)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.sql.Date.getDeclaredMethods()
without it being registered for runtime reflection. Add java.sql.Date.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector._addMemberMethods(AnnotatedMethodCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector.collect(AnnotatedMethodCollector.java:49)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.util.Date.getDeclaredMethods()
without it being registered for runtime reflection. Add java.util.Date.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector._addMemberMethods(AnnotatedMethodCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector.collect(AnnotatedMethodCollector.java:54)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.io.Serializable.getDeclaredMethods()
without it being registered for runtime reflection. Add java.io.Serializable.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector._addMemberMethods(AnnotatedMethodCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector.collect(AnnotatedMethodCollector.java:54)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.lang.Cloneable.getDeclaredMethods()
without it being registered for runtime reflection. Add java.lang.Cloneable.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector._addMemberMethods(AnnotatedMethodCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector.collect(AnnotatedMethodCollector.java:54)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.lang.Comparable.getDeclaredMethods()
without it being registered for runtime reflection. Add java.lang.Comparable.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector._addMemberMethods(AnnotatedMethodCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedMethodCollector.collect(AnnotatedMethodCollector.java:54)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.sql.Date.getDeclaredConstructors()
without it being registered for runtime reflection. Add java.sql.Date.getDeclaredConstructors() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredConstructors(DynamicHub.java:1264)
com.fasterxml.jackson.databind.util.ClassUtil.getConstructors(ClassUtil.java:1345)
com.fasterxml.jackson.databind.introspect.AnnotatedCreatorCollector._findPotentialConstructors(AnnotatedCreatorCollector.java:115)
com.fasterxml.jackson.databind.introspect.AnnotatedCreatorCollector.collect(AnnotatedCreatorCollector.java:70)
org.graalvm.nativeimage.MissingReflectionRegistrationError: The program tried to reflectively access
java.sql.Date.getDeclaredMethods()
without it being registered for runtime reflection. Add java.sql.Date.getDeclaredMethods() to the reflection metadata to solve this problem. See https://www.graalvm.org/latest/reference-manual/native-image/metadata/#reflection for help.
java.base@23.0.1/java.lang.Class.getDeclaredMethods(DynamicHub.java:1258)
com.fasterxml.jackson.databind.util.ClassUtil.getClassMethods(ClassUtil.java:1300)
com.fasterxml.jackson.databind.introspect.AnnotatedCreatorCollector._findPotentialFactories(AnnotatedCreatorCollector.java:197)
com.fasterxml.jackson.databind.introspect.AnnotatedCreatorCollector.collect(AnnotatedCreatorCollector.java:71)
Describe the solution you'd like
Ideally, we should avoid dynamically resolving this information for java.*
classes.
Usage example
No response
Additional context
No response
Comment From: cowtowncoder
Hmmmh. Ok, now I remember the challenge: although there is existing mechanism to avoid annotation introspection for JDK classes (although only applied to JDK collection types), actual introspection of declared fields etc is still performed. So that'd need to be changed and it's bigger change.
Preventing this traversal would also prevent use of mix-in annotations on JDK types.
I'll probably make this a MapperFeature
but one that is opt-out (i.e. default being to skip introspection).
Comment From: cowtowncoder
Another question: once I get to implement something, is there some tool I could use to see if access is violated? It'd be good for me to verify changes locally before asking others to check.
Comment From: cowtowncoder
Ok so just removing --add-opens
from pom.xml
will not expose these (although it does trigger 4 failures, one for EnumSet
another for EnumMap
type detection, 2 others for trying to force access to JDK internal collections -- I fixed 2 others that happened).
Is there maybe another compiler option to get warnings? Or is this with Graal processing?
@gsmet I'd be happy to work a bit on this now but need to see warnings to verify calls are no longer made. (ideally there'd be tests too of course but first things first).
Comment From: gsmet
@cowtowncoder sorry, things have been quite chaotic lately. Let me try to assemble a simple reproducer for you.
Comment From: gsmet
@cowtowncoder the feedback loop is not exactly fast as it requires building a native executable but the following reproducer reproduces the issue:
See the README in it for detailed instructions. Let me know if you encounter any issue.
You should be able to override Jackson in the POM to point to a SNAPSHOT.
Comment From: cowtowncoder
Thank you @gsmet !
Comment From: cowtowncoder
Ok after couple of small tweaks (figuring out dir ref to GraalVM etc), got build going. So I think this is fine.
EDIT: and after building, running service with given settings does indeed report expected warnings.
Comment From: cowtowncoder
@gsmet Trying to work on this again, now that 2.19.0 is released. Noticing that I can only build against 2.18.*; attempts to building against 2.19(.0) fail due to one thing added to jackson-annotations
in 2.19.
Would it be easy to indicate if I can make local modifications to reproducer (aside from jackson-databind version in pom.xml) -- or if need be, create new reproducer zip? Apologies for extra work, but I'd like to verify if my changes help -- now that I can see what specifically is printing warnings.
Comment From: cowtowncoder
Quick note: been investigating this, learning much more about the challenges.
There are multiple ways to (try to) achieve this, f.ex:
- Reduce introspection of accessors at low-level, for JDK types. Could exclude Fields without problems, but for Methods not doable without allow-listing some types (getters relied on for some serialization of
StackTraceElement
,ThreadGroup
, as-POJO-serialization forOptional
-- see #5119 ) - Try to do (de)serializer lookup by providers/defaults before (most?) introspection, to prevent introspection for "known" JDK types (ones with handlers)
Of these, both seem difficult to safely do for 2.x, so I will focus more on 3.0; but if it happened that a safe set of changes were found, would backport into 2.x (2.20).
FWTW, I am thinking (2) would be most beneficial route but we'll see.
Comment From: cowtowncoder
Ok some incremental progress, for 3.0 (.0-rc5): now for deserialization side, full introspection is avoided for deserialization of "well-known" types (ones for which Jackson or modules has explicitly registered deserializer) such as java.sql.Date
used in reproduction.
For serialization calls are deferred as well, with the unfortunate exception for @JsonValue
introspection which must be done for accessors (Fields, Getters).
To avoid that, different approach is needed; will need to think about that.
Note, though, that there is parallel effort via #5149 which would:
- Allow for explicit disabling of Field/Method/Constructor introspection for specific categories of classes (like Core JDK) AND
- Would be implemented for 2.x (2.20)
and which I hope to work on. But haven't had much progress yet (at idea/design stage).
Comment From: cowtowncoder
Phew! I figured out how to order things to avoid Method introspection for serialization of explicitly supported types: as I mentioned earlier this is for 3.0.0. Will mark this as closed.
Comment From: cowtowncoder
@gsmet Given that work to re-order things is based on working around specific type (java.sql.Date
in this case), there are likely other paths that would still trigger same issue. So I would be interesting in finding more if you happen to have time. These can be in 2.x (whichever version); specifically thinking of List
s and Map
s -- scalar types are probably covered.