Jackson Databind Jackson 3 (rc7) deserialization of class with public final Field quietly fails

Search before asking

• [x] I searched in the issues and found nothing similar.

Describe the bug

The attached class works correctly on Jackson 2 - but errors on Jackson 3.

Version Information

jackson-databind-3.0.0-rc7

Reproduction

Class:

package j3bug;

import java.util.HashMap;
import java.util.Map;

public class DateTimeParserState {
    public int sampleCount;
    public int nullCount;
    public int blankCount;
    public int invalidCount;
    public final Map<String, Integer> results;

    public DateTimeParserState() {
        results = new HashMap<>();
    }
}

Driver:

package j3bug;

import java.util.Locale;

import tools.jackson.core.StreamReadFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;

public abstract class J3bug {

    public static void main(String[] args) {
        ObjectMapper mapper = JsonMapper.builder().enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION).build();
        DateTimeParserState state = new DateTimeParserState();
        state.results.put("hello", 1);
        String serialized = mapper.writeValueAsString(state);
        DateTimeParserState post = mapper.readValue(serialized, DateTimeParserState.class);
        if (post.results.size() != 1) {
            System.err.println("Houston, we have a problem!");
            System.exit(1);
        }
    }
}

Expected behavior

Expected Map to be restored to its former glory (i.e. one entry) when deserialized.

Additional context

No response

Comment From: pjfanning

Can you provide more details? * what is the input JSON? Include it instead of forcing someone to generate it themselves. Have you checked if the JSON that is serialized id what you expect? I've seen people report issues about deserialization where the real reason is that serialization changed and the new output of serialization causes issues when deserializing. * what is the exception when deserializing?

Comment From: tsegall

Apologies for not being clearer. The above program, if executed with Jackson 2 works. If executed with Jackson 3 it does not.

This is not an attempt to take a serialized Jackson 2 string and deserialize it in Jackson 3.

As per the program above - when you run this with Jackson 3 and add one element to the array:

The serialization of the objects is {"blankCount":0,"invalidCount":0,"nullCount":0,"results":{"hello":1},"sampleCount":0}

Post readValue() the object is {"blankCount":0,"invalidCount":0,"nullCount":0,"results":{},"sampleCount":0}

That is it is missing the results Map.

There is no exception when the program is run - just the wrong result.

Comment From: pjfanning

Thanks @tsegall for the extra information. I don't have time in the next few hours to look at this but may get back to it later and someone else may be able to help too. One interesting experiment would be be to try making the 'results' field non-final. There may also be options around adding Jackson annotations. I'm not saying that we want to keep the new behaviour that you are seeing but it might be interesting to find out what works and what doesn't.

Comment From: tsegall

@pjfanning As you clearly suspected :-) it is the final that is causing the issue. Without the final the program works as expected.

Comment From: pjfanning

@tsegall would you consider adding a constructor to your class that sets all the fields? Having such a public constructor means that the deserializer can use a public API to create the class instance instead of relying on reflection APIs that ignore private and final markers on fields. Java is making it harder to use the Reflection backdoors for security reasons.

Comment From: cowtowncoder

If it's about final Field, I think it's due to

https://github.com/FasterXML/jackson-databind/blob/3.x/src/main/java/tools/jackson/databind/MapperFeature.java#L83

change via https://github.com/FasterXML/jackson-databind/issues/4552

where default for MapperFeature.ALLOW_FINAL_FIELDS_AS_MUTATORS was changed to false (as mentioned on https://github.com/FasterXML/jackson-future-ideas/wiki/JSTEP-2.

So it is possible to just enable this feature. But one caveat is that this feature may not work in newer JVMs.

Comment From: cowtowncoder

One thing that sounds and seems wrong is the lack of exception: but this is due to another defaults change:

https://github.com/FasterXML/jackson-databind/issues/493

which is against my personal preferences, but strongly supported by user base it seems. So basically now unknown (since final fields are not considered as Mutators) properties are just ignored by default. This default can be changed for mapper too of course.

So I think this is what is going on here.

Comment From: tsegall

@cowtowncoder as @pjfanning suggested I added a public constructor that sets all the fields - but this did not help (it was not invoked).

So it does not sound like I should enable MapperFeature.ALLOW_FINAL_FIELDS_AS_MUTATORS as this is going to be a bad idea moving forward.

So it seems I can either remove the final or make results private and add getters and setters.

Thoughts?

Comment From: cowtowncoder

@tsegall Please share your code for constructor add. It will work, but depending on details you may need to annotate constructor with @JsonCreator.

Also, for testing at least, consider enabling DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES as that often uncovers naming mismatches.

As to enabling MapperFeature.ALLOW_FINAL_FIELDS_AS_MUTATORS, yeah, it is ideally not relied on as it depends on specific quirk of JVM allowing mutation of "final" fields for brief period of time after insantiation (to support JDK serialization I think).

Comment From: tsegall

@cowtowncoder I added the following

        public DateTimeParserState(final int sampleCount, final int nullCount, final int blankCount, final int invalidCount, final Map<String, Integer> results) {
                System.err.println("Contructor invoked");
                this.sampleCount = sampleCount;
                this.nullCount = nullCount;
                this.blankCount = blankCount;
                this.invalidCount = invalidCount;
                this.results = results;
        }

Interestingly if I do as you suggest and add this DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES then I do get an error as opposed to a silent failure.

It seems to be ignoring the results property because it is final. Note: If I remove the final it will call the new constructor above and all works.

Exception in thread "main" tools.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized property "results" (class j3bug.DateTimeParserState), not marked as ignorable (4 known properties: "blankCount", "invalidCount", "nullCount", "sampleCount"]) at [Source: (String)"{"blankCount":0,"invalidCount":0,"nullCount":0,"results":{"hello":1},"sampleCount":0}"; line: 1, column: 59] (through reference chain: j3bug.DateTimeParserState["results"]) at tools.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51) at tools.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:1269) at tools.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:2001) at tools.jackson.databind.deser.bean.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1724) at tools.jackson.databind.deser.bean.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1702) at tools.jackson.databind.deser.bean.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:545) at tools.jackson.databind.deser.bean.BeanDeserializer.deserialize(BeanDeserializer.java:200) at tools.jackson.databind.deser.DeserializationContextExt.readRootValue(DeserializationContextExt.java:265) at tools.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2688) at tools.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1600) at j3bug.J3bug.main(J3bug.java:23)

Comment From: cowtowncoder

@tsegall Hmmm. final should not matter for constructor. Let me see if I can reproduce the problem.

Comment From: cowtowncoder

@tsegall Cannot replicate, see #5255. Will try to change to actually use Map, should not really matter.

So need help reproducing the issue.

Comment From: tsegall

@cowtowncoder The following reproduces the issue for me.

Class:

package j3bug;

import java.util.HashMap;
import java.util.Map;

public class DateTimeParserState {
    public int sampleCount;
    public int nullCount;
    public int blankCount;
    public int invalidCount;
    public final Map<String, Integer> results;

    public DateTimeParserState(final int sampleCount, final int nullCount, final int blankCount, final int invalidCount, final Map<String, Integer> results) {
        System.err.println("Contructor invoked");
        this.sampleCount = sampleCount;
        this.nullCount = nullCount;
        this.blankCount = blankCount;
        this.invalidCount = invalidCount;
        this.results = results;
    }

    public DateTimeParserState() {
        System.err.println("Contructor small invoked");
        results = new HashMap<>();
    }
}

Driver:

package j3bug;

import java.util.HashMap;
import java.util.Locale;
import java.util.Map;

import tools.jackson.core.StreamReadFeature;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.json.JsonMapper;

public abstract class J3bug {

    public static void main(String[] args) {
        ObjectMapper mapper = JsonMapper.builder().enable(StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION)
            .enable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
            .build();
        final Map<String, Integer> notMuch = new HashMap<>();
        notMuch.put("hello", 1);
        DateTimeParserState state = new DateTimeParserState(0, 0, 0, 0, notMuch);
        String serialized = mapper.writeValueAsString(state);
        System.err.println("pre serialized = " + serialized);
        DateTimeParserState post = mapper.readValue(serialized, DateTimeParserState.class);
        System.err.println("post serialized = " + mapper.writeValueAsString(post));
        if (post.results.size() != 1) {
            System.err.println("Houston, we have a problem!");
            System.exit(1);
        }
    }
}