Jackson Databind Serialize and deserialize enum as object with property typing

Describe your Issue

Hello.

I'm not sure is this a bug, not implemented feature or just my lack of knowledge, so just want to ask a help :)

Jackson version: 2.18.3

I have to deal with some protocol, that has explicit typing with @class property in every object and enum, but not in arrays/primitives - like this:

[{
  "@class": "com.protocol.Parameter",
  "id": "1",
  "value": 1
},{
  "@class": "com.protocol.Parameter",
  "id": "2",
  "value": {
    "@class": "com.protocol.BiometricType",
    "value": "FACE_ID"
  }
},{
  "@class": "com.protocol.Parameter",
  "id": "3",
  "value": {
    "@class": "com.protocol.MyClass",
    "classField": "classFieldValue"
  }
}]

Also I have a following classes:

class Parameter {
    String id;
    Object value;    // can be any primitive/object/enum
}

enum BiometricType {
    TOUCH_ID, FACE_ID
}

Then I'm trying to serialize object to json:

// custom TypeResolverBuilder to exclude arrays from typing
public class OnlyObjectsTypeResolverBuilder extends StdTypeResolverBuilder {

    public OnlyObjectsTypeResolverBuilder(String propName) {
        super(JsonTypeInfo.Id.CLASS, JsonTypeInfo.As.PROPERTY, propName);
    }

    @Override
    public TypeDeserializer buildTypeDeserializer(DeserializationConfig config, JavaType baseType, Collection<NamedType> subtypes) {
        return useForType(baseType) ? super.buildTypeDeserializer(config, baseType, subtypes) : null;
    }

    @Override
    public TypeSerializer buildTypeSerializer(SerializationConfig config, JavaType baseType, Collection<NamedType> subtypes) {
        return useForType(baseType) ? super.buildTypeSerializer(config, baseType, subtypes) : null;
    }

    public boolean useForType(JavaType t) {
        return !t.isPrimitive() && !t.isArrayType()
    }
}

ObjectMapper objectMapper = new ObjectMapper()
  .setDefaultTyping(new OnlyObjectsTypeResolverBuilder("@class"));

objectMapper.writeValueAsString(new Parameter("type", BiometricType.FACE_ID));

and get this:

{
  "@class": "com.protocol.Parameter",
  "id": "type",
  "value": [
    "com.protocol.BiometricType",
    "FACE_ID"
  ]
}

Looks fine, but I need an object, that surrounds enum, not array (objects and primitives are also fine by default, so just skip them, I have problems only with enums).

Ok, let's try to add @JsonFormat(shape = JsonFormat.Shape.OBJECT) to the enum:

{
  "@class": "com.protocol.Parameter",
  "id": "type",
  "value": {
    "@class": "com.protocol.BiometricType"
  }
}

Where enum value has gone? Is this a bug or intended?

Finally i give up to the custom serializer:

private static class EnumAsObjectSerializer extends JsonSerializer<Enum> {

    @Override
    public void serialize(Enum value, JsonGenerator gen, SerializerProvider serializers) {
        throw new UnsupportedOperationException("should never be called");
    }

    @Override
    public void serializeWithType(Enum value, JsonGenerator gen, SerializerProvider serializers, TypeSerializer typeSer)
        throws IOException {
        WritableTypeId typeIdDef = typeSer.writeTypePrefix(gen, typeSer.typeId(value, JsonToken.START_OBJECT));
        gen.writeStringField("value", value.name());
        typeSer.writeTypeSuffix(gen, typeIdDef);
    }
}

This works fine, but is this the only way or I missed something standart?

Next I want to deserialize this json back:

objectMapper.readValue("""
{
  "@class": "com.protocol.Parameter",
  "id": "type",
  "value": {
    "@class": "com.protocol.BiometricType",
    "value": "FACE_ID"
  }
}
""", Object.class)

And I get this error (with and without @JsonFormat annotation)

Cannot deserialize value of type `com.protocol.BiometricType` from Object value (token `JsonToken.FIELD_NAME`)
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `com.protocol.BiometricType` from Object value (token `JsonToken.FIELD_NAME`)
(through reference chain: com.protocol.Parameter["value"])
com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1767)
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1541)
com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1446)
com.fasterxml.jackson.databind.deser.std.EnumDeserializer._deserializeOther(EnumDeserializer.java:457)
com.fasterxml.jackson.databind.deser.std.EnumDeserializer.deserialize(EnumDeserializer.java:292)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromAny(AsPropertyTypeDeserializer.java:240)
com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserializeWithType(UntypedObjectDeserializerNR.java:112)
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:138)
com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:310)
com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:215)
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:170)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:136)
com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromAny(AsPropertyTypeDeserializer.java:240)
com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserializeWithType(UntypedObjectDeserializerNR.java:112)
com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:74)
com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4931)
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3868)
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3836)

Looks like deserializer doesn't know what to do with value field, ok, that is expected, because i used custom serializer. Let's try with custom deserializer:

private static class EnumAsObjectDeserializer extends JsonDeserializer<Enum> {
    /*@Override
    public Object deserializeWithType(JsonParser p, DeserializationContext ctxt, TypeDeserializer typeDeserializer) {
        // at first i expected to implement this method as opposite to `serializeWithType`, but this method is never called.
        // did i missed smthng?
    }*/

    @Override
    public Enum deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
        JsonNode node = p.getCodec().readTree(p);

        // this is called fine, but `@class` is unavailable here,
        // because it was already read in `AsPropertyTypeDeserializer._deserializeTypedForId()`
        String className = node.get("@class").asText();
        String value = node.get("value").asText();

        try {
            Class<Enum> enumClass = (Class<Enum>) Class.forName(className);
            return Enum.valueOf(enumClass, value);
        }
        catch (ClassNotFoundException e) {
            throw new IOException("Could not find enum class: " + className, e);
        }
    }
}

So: - I can't access the type of enum in the plain deserialize() method - deserializeWithType() is never called

The only way I found is to enable typeIdVisible in StdTypeResolverBuilder so code here will merge @class property back to parser. But this looks like a not optimal solution, because: - enables type info backmerging for every object - requires to disable DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES - can potentially break other deserializers that not expect type info

So is there any other way to implement (de)serializing of such json? Preferably without annotations, just with ObjectMapper configurations, because I can't edit the protocol classes.

Thanks!

Comment From: cowtowncoder

Ok, what you have is very complicated set up. :) (and yes, I understand it's due to existing format to support and not your choice).

But just one quick note: to include "external" Type Id, use JsonTypeInfo.As.EXTERNAL_PROPERTY, not JsonTypeInfo.As.PROPERTY (which is "internal"). Ideally you wouldn't need to play with StdTypeResolverBuilder as that's complicated enough to be difficult to reason about.

Another tiny thing (which doesn't greatly matter), this:

   JsonNode node = p.getCodec().readTree(p);

can be replaced with

  JsonNode node = ctxt.readTree(p);

which works bit better (but is not causing issues here)

Comment From: badmannersteam

Thanks for response!

But just one quick note: to include "external" Type Id, use JsonTypeInfo.As.EXTERNAL_PROPERTY, not JsonTypeInfo.As.PROPERTY (which is "internal").

Maybe I understand something wrong, but my @class property is on the same level as the other properties:

{
  "@class": "com.protocol.Parameter",
  "id": "type",
  "value": {
    "@class": "com.protocol.BiometricType",
    "value": "FACE_ID"
  }
}

or for enum this can be qualified as "external"?

{
  "@class": "com.protocol.BiometricType",
  "value": "FACE_ID"
}

btw, i got an error for this code:

new ObjectMapper()
    // the same code as above, just `PROPERTY` replaced with `EXTERNAL_PROPERTY`
    .setDefaultTyping(new OnlyObjectsTypeResolverBuilder(JsonTypeInfo.As.EXTERNAL_PROPERTY, "@class"))
    .readValue("""
        {
          "@class": "com.protocol.Parameter",
          "id": "type",
          "value": {
            "@class": "com.protocol.BiometricType",
            "value": "FACE_ID"
          }
        }
        """, Object.class);

Unexpected token (START_OBJECT), expected START_ARRAY: need Array value to contain `As.WRAPPER_ARRAY` type information for class java.lang.Object
com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
com.fasterxml.jackson.databind.DeserializationContext.wrongTokenException(DeserializationContext.java:1914)
com.fasterxml.jackson.databind.DeserializationContext.reportWrongTokenException(DeserializationContext.java:1699)
com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._locateTypeId(AsArrayTypeDeserializer.java:150)
com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer._deserialize(AsArrayTypeDeserializer.java:99)
com.fasterxml.jackson.databind.jsontype.impl.AsArrayTypeDeserializer.deserializeTypedFromAny(AsArrayTypeDeserializer.java:74)
com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserializeWithType(UntypedObjectDeserializerNR.java:112)
com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:74)
com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4931)
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3868)
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3836)

Comment From: cowtowncoder

Ah, you are using Default Typing. Never mind then, this is not an option unfortunately.

I also missed the part that structure is sort of mix: there is id along with @class, not just value.

EXISTING_PROPERTY also needs to be defined for a property of a parent POJO (to bind together "value" property and "type id".

So just ignore my suggestion, I was not reading through issue carefully.

Comment From: cowtowncoder

On:

Ok, let's try to add @JsonFormat(shape = JsonFormat.Shape.OBJECT) to the enum:

this will basically change logic to treat Enum as if it was POJO, that is, serialize logical properties (visible Getter methods and/or Fields). That is, OBJECT here is for "Java Object access (not Enum)", more than "output as JSON Object" (although it does achieve that too). This is quite confusing annotation -- Jackson 3.0 adds Shape.POJO to try to make it slightly confusing.

Comment From: badmannersteam

I finally resolved my deserialization issues with the help of custom AsPropertyTypeDeserializer, that handles enums in _deserializeTypedForId (one level higher than custom deserializers).

Hope this final code will help someone.

// serialization
public class EnumAsObjectModule extends SimpleModule {

    public EnumAsObjectModule() {
        addSerializer(Enum.class, new EnumAsObjectSerializer());
    }

    public static class EnumAsObjectSerializer extends JsonSerializer<Enum> {

        @Override
        public void serialize(Enum value, JsonGenerator gen, SerializerProvider serializers) {
            throw new UnsupportedOperationException("should never be called");
        }

        @Override
        public void serializeWithType(Enum value, JsonGenerator gen, SerializerProvider serializers, TypeSerializer typeSer)
            throws IOException {
            WritableTypeId typeIdDef = typeSer.writeTypePrefix(gen, typeSer.typeId(value, JsonToken.START_OBJECT));
            gen.writeStringField("value", value.name());
            typeSer.writeTypeSuffix(gen, typeIdDef);
        }
    }
}


// typing and deserialization
public class ObjectsAndEnumsTypeResolverBuilder extends StdTypeResolverBuilder {

    public ObjectsAndEnumsTypeResolverBuilder(String propName) {
        super(JsonTypeInfo.Id.CLASS, JsonTypeInfo.As.PROPERTY, propName);
    }

    @Override
    public TypeSerializer buildTypeSerializer(SerializationConfig config, JavaType baseType,
        Collection<NamedType> subtypes) {
        return useForType(baseType) ? super.buildTypeSerializer(config, baseType, subtypes) : null;
    }

    @Override
    public TypeDeserializer buildTypeDeserializer(DeserializationConfig config, JavaType baseType,
        Collection<NamedType> subtypes) {
        return useForType(baseType) ? buildEnumAwareAsPropertyTypeDeserializer(config, baseType, subtypes) : null;
    }

    private boolean useForType(JavaType t) {
        if (t.isPrimitive() || t.isArrayType()) {
            return false;
        } else {
            while (t.isReferenceType()) {
                t = t.getReferencedType();
            }
            return !t.isFinal() && !TreeNode.class.isAssignableFrom(t.getRawClass());
        }
    }

    private EnumAwareAsPropertyTypeDeserializer buildEnumAwareAsPropertyTypeDeserializer(DeserializationConfig config, JavaType baseType, Collection<NamedType> subtypes) {
        PolymorphicTypeValidator subTypeValidator = verifyBaseTypeValidity(config, baseType);
        TypeIdResolver idRes = idResolver(config, baseType, subTypeValidator, subtypes, false, true);
        return new EnumAwareAsPropertyTypeDeserializer(baseType, idRes, _typeProperty, _typeIdVisible,
            defineDefaultImpl(config, baseType), _includeAs, _strictTypeIdHandling(config, baseType));
    }

    public static class EnumAwareAsPropertyTypeDeserializer extends AsPropertyTypeDeserializer {

        public EnumAwareAsPropertyTypeDeserializer(EnumAwareAsPropertyTypeDeserializer src, BeanProperty property) {
            super(src, property);
        }

        public EnumAwareAsPropertyTypeDeserializer(JavaType bt, TypeIdResolver idRes, String typePropertyName,
            boolean typeIdVisible, JavaType defaultImpl, JsonTypeInfo.As inclusion, boolean strictTypeIdHandling) {
            super(bt, idRes, typePropertyName, typeIdVisible, defaultImpl, inclusion, strictTypeIdHandling);
        }

        @Override
        public TypeDeserializer forProperty(BeanProperty prop) {
            return (prop == _property) ? this : new EnumAwareAsPropertyTypeDeserializer(this, prop);
        }

        @Override
        protected Object _deserializeTypedForId(JsonParser p, DeserializationContext ctxt, TokenBuffer tb, String typeId)
            throws IOException {

            JsonDeserializer<Object> deser = _findDeserializer(ctxt, typeId);
            if (p.currentToken() != JsonToken.END_OBJECT) {
                p.nextToken();
            }

            if (deser.handledType().isEnum()) {
                JsonNode node = ctxt.readTree(p);
                String value = node.get("value").asText();
                return Enum.valueOf((Class<Enum>) deser.handledType(), value);
            } else {
                return deser.deserialize(p, ctxt);
            }
        }
    }
}


// usage
ObjectMapper objectMapper = new ObjectMapper()
        .setDefaultTyping(new ObjectsAndEnumsTypeResolverBuilder("@class"))
        .registerModule(new EnumAsObjectModule());

objectMapper.writeValueAsString(new Parameter("id", BiometricType.FACE_ID));

objectMapper.readValue("""
        {
          "@class": "com.protocol.Parameter",
          "id": "id",
          "value": {
            "@class": "com.protocol.BiometricType",
            "value": "FACE_ID"
          }
        }
        """, Object.class);

Comment From: cowtowncoder

Thank you for sharing @badmannersteam -- and glad to know you were able to figure out a way to solve the issue.