Search before asking
- [x] I searched in the issues and found nothing similar.
Describe the bug
It seems like @JsonPOJOBuilder doesn't play nice with @JsonAnySetter. In the following example, the virtualized property "attributes' (which should contain all properties not specified in the POJO), is null when deserialized, even though additional properties exist in the input JSON.
Version Information
2.18.4/2.19.0/2.19.1
Reproduction
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import com.fasterxml.jackson.annotation.JsonAnyGetter;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class TestTest {
@Test
public void test() throws JsonProcessingException {
final ObjectMapper mapper = new ObjectMapper();
final String json = "{\"name\":\"name\",\"items\":[{\"type\":\"type\",\"key1\":\"value1\",\"key2\":\"value2\"}]}";
final CollectionType serialized = mapper.readValue(json, CollectionType.class);
assertNotNull(serialized.getItems().get(0).getAttributes());
assertEquals("value1", serialized.getItems().get(0).getAttributes().get("key1"));
assertEquals("value2", serialized.getItems().get(0).getAttributes().get("key2"));
}
@com.fasterxml.jackson.databind.annotation.JsonDeserialize(builder = TestTest.CollectionType.CollectionTypeBuilder.class)
public static class CollectionType {
private String name;
private List<Item> items;
@com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder(withPrefix = "", buildMethodName = "build")
public static class CollectionTypeBuilder {
private String name;
private List<Item> items;
CollectionTypeBuilder() {
}
/**
* @return {@code this}.
*/
public TestTest.CollectionType.CollectionTypeBuilder name(final String name) {
this.name = name;
return this;
}
/**
* @return {@code this}.
*/
public TestTest.CollectionType.CollectionTypeBuilder items(final List<Item> items) {
this.items = items;
return this;
}
public TestTest.CollectionType build() {
return new TestTest.CollectionType(this.name, this.items);
}
@Override
public java.lang.String toString() {
return "TestTest.CollectionType.CollectionTypeBuilder(name=" + this.name + ", items=" + this.items + ")";
}
}
public static TestTest.CollectionType.CollectionTypeBuilder builder() {
return new TestTest.CollectionType.CollectionTypeBuilder();
}
public String getName() {
return this.name;
}
public List<Item> getItems() {
return this.items;
}
public void setName(final String name) {
this.name = name;
}
public void setItems(final List<Item> items) {
this.items = items;
}
@Override
public boolean equals(final java.lang.Object o) {
if (o == this) return true;
if (!(o instanceof TestTest.CollectionType)) return false;
final TestTest.CollectionType other = (TestTest.CollectionType) o;
if (!other.canEqual((java.lang.Object) this)) return false;
final java.lang.Object this$name = this.getName();
final java.lang.Object other$name = other.getName();
if (this$name == null ? other$name != null : !this$name.equals(other$name)) return false;
final java.lang.Object this$items = this.getItems();
final java.lang.Object other$items = other.getItems();
if (this$items == null ? other$items != null : !this$items.equals(other$items)) return false;
return true;
}
protected boolean canEqual(final java.lang.Object other) {
return other instanceof TestTest.CollectionType;
}
@Override
public int hashCode() {
final int PRIME = 59;
int result = 1;
final java.lang.Object $name = this.getName();
result = result * PRIME + ($name == null ? 43 : $name.hashCode());
final java.lang.Object $items = this.getItems();
result = result * PRIME + ($items == null ? 43 : $items.hashCode());
return result;
}
@Override
public java.lang.String toString() {
return "TestTest.CollectionType(name=" + this.getName() + ", items=" + this.getItems() + ")";
}
public CollectionType() {
}
public CollectionType(final String name, final List<Item> items) {
this.name = name;
this.items = items;
}
}
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonDeserialize(builder = TestTest.Item.ItemBuilder.class)
public static class Item {
private String type;
@JsonAnyGetter
@JsonAnySetter
private Map<String, Object> attributes;
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder(withPrefix = "", buildMethodName = "build")
public static class ItemBuilder {
private String type;
private Map<String, Object> attributes;
ItemBuilder() {
}
/**
* @return {@code this}.
*/
public TestTest.Item.ItemBuilder type(final String type) {
this.type = type;
return this;
}
/**
* @return {@code this}.
*/
public TestTest.Item.ItemBuilder attributes(final Map<String, Object> attributes) {
this.attributes = attributes;
return this;
}
public TestTest.Item build() {
return new TestTest.Item(this.type, this.attributes);
}
@Override
public java.lang.String toString() {
return "TestTest.Item.ItemBuilder(type=" + this.type + ", attributes=" + this.attributes + ")";
}
}
public static TestTest.Item.ItemBuilder builder() {
return new TestTest.Item.ItemBuilder();
}
public String getType() {
return this.type;
}
public Map<String, Object> getAttributes() {
return this.attributes;
}
public void setType(final String type) {
this.type = type;
}
public void setAttributes(final Map<String, Object> attributes) {
this.attributes = attributes;
}
@Override
public boolean equals(final java.lang.Object o) {
if (o == this) return true;
if (!(o instanceof TestTest.Item)) return false;
final TestTest.Item other = (TestTest.Item) o;
if (!other.canEqual((java.lang.Object) this)) return false;
final java.lang.Object this$type = this.getType();
final java.lang.Object other$type = other.getType();
if (this$type == null ? other$type != null : !this$type.equals(other$type)) return false;
final java.lang.Object this$attributes = this.getAttributes();
final java.lang.Object other$attributes = other.getAttributes();
if (this$attributes == null ? other$attributes != null : !this$attributes.equals(other$attributes)) return false;
return true;
}
protected boolean canEqual(final java.lang.Object other) {
return other instanceof TestTest.Item;
}
@Override
public int hashCode() {
final int PRIME = 59;
int result = 1;
final java.lang.Object $type = this.getType();
result = result * PRIME + ($type == null ? 43 : $type.hashCode());
final java.lang.Object $attributes = this.getAttributes();
result = result * PRIME + ($attributes == null ? 43 : $attributes.hashCode());
return result;
}
@Override
public java.lang.String toString() {
return "TestTest.Item(type=" + this.getType() + ", attributes=" + this.getAttributes() + ")";
}
public Item() {
}
public Item(final String type, final Map<String, Object> attributes) {
this.type = type;
this.attributes = attributes;
}
}
}
Expected behavior
The additional properties are deserialized
If the following is replaced, it works:
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonDeserialize(builder = TestTest.Item.ItemBuilder.class)
public static class Item {
private String type;
@JsonAnyGetter
@JsonAnySetter
private Map<String, Object> attributes;
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder(withPrefix = "", buildMethodName = "build")
public static class ItemBuilder {
with
@JsonIgnoreProperties(ignoreUnknown = true)
public static class Item {
private String type;
@JsonAnyGetter
@JsonAnySetter
private Map<String, Object> attributes;
public static class ItemBuilder {
Additional context
No response
Comment From: pjfanning
@mikethecalamity Thanks for the detailed POC. Unfortunately, we don't test 3rd party libs. The only test cases we accept are pure Java with no external lib dependencies like Lombok.
Comment From: pjfanning
I don't know much about the internals of Lombok and I haven't used it in years. I read that lombok.Generated is not supposed to do anything that it is some sort of marker annotation. Would it be possible to reproduce the issue using a dummy marker annotation?
Comment From: mikethecalamity
@pjfanning That is correct, the lombok.Generated is just a marker annotation, but regardless, I've removed every third party dependency (except JUnit). The issue is still presenting.
Comment From: pjfanning
@mikethecalamity I put your POC in https://github.com/pjfanning/jackson-lombok-test - the name is wrong now that Lombok is not needed. I modified it a bit to work in Java 8 because this 2.x branch needs our tests to compile in Java 8. I have verified that the test fails as a you describe.
Comment From: pjfanning
I am not an expert on the Jackson annotations and how they affect jackson-databind. @cowtowncoder, the main contributor, to this project is on vacation. One observation is that if you remove the JsonAnyGetter annotation, this test passes. Likewise, the JsonAnySetter doesn't seem to be required either. The test passes if you remove both annotations. I agree that this issue is a regression. I'm just highlighting that jackson-databind is often a lot easier to use if you don't resort to using lots of Jackson annotations. It is close to impossible to test all the possible permutations of using Jackson annotations together.
Comment From: mikethecalamity
I can't do that because it would change the structure of the JSON. The attributes property is supposed to be a virtualized property containing all JSON properties not directly specified.
You could add this to the test to verify:
assertEquals("{\"name\":\"name\",\"items\":[{\"type\":\"type\",\"key1\":\"value1\",\"key2\":\"value2\"}]}", json);
I did find if I change:
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonDeserialize(builder = TestTest.Item.ItemBuilder.class)
public static class Item {
private String type;
@JsonAnyGetter
@JsonAnySetter
private Map<String, Object> attributes;
@JsonIgnoreProperties(ignoreUnknown = true)
@com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder(withPrefix = "", buildMethodName = "build")
public static class ItemBuilder {
private String type;
private Map<String, Object> attributes;
to
@JsonIgnoreProperties(ignoreUnknown = true)
public static class Item {
private String type;
@JsonAnyGetter
@JsonAnySetter
private Map<String, Object> attributes;
public static class ItemBuilder {
private String type;
private Map<String, Object> attributes;
Then it works, I'm curious if something broke with the JsonPOJOBuilder and how it interacts with the JsonAnyGetter/JsonAnySetter
Comment From: mikethecalamity
This is probably a better test:
final ObjectMapper mapper = new ObjectMapper();
final String json = "{\"name\":\"name\",\"items\":[{\"type\":\"type\",\"key1\":\"value1\",\"key2\":\"value2\"}]}";
final CollectionType serialized = mapper.readValue(json, CollectionType.class);
assertNotNull(serialized.getItems().get(0).getAttributes());
assertEquals("value1", serialized.getItems().get(0).getAttributes().get("key1"));
assertEquals("value2", serialized.getItems().get(0).getAttributes().get("key2"));
Comment From: pjfanning
This is probably a better test:
final ObjectMapper mapper = new ObjectMapper(); final String json = "{\"name\":\"name\",\"items\":[{\"type\":\"type\",\"key1\":\"value1\",\"key2\":\"value2\"}]}"; final CollectionType serialized = mapper.readValue(json, CollectionType.class); assertNotNull(serialized.getItems().get(0).getAttributes()); assertEquals("value1", serialized.getItems().get(0).getAttributes().get("key1")); assertEquals("value2", serialized.getItems().get(0).getAttributes().get("key2"));
@mikethecalamity the sample in the description has "key", "value", "key2", "value2"
Comment From: pjfanning
with the description in the test case, I notice that the serialized json has changed between Jackson 2.18.4 and 2.19.1
2.18.4
{"name":"name","items":[{"type":"type","attributes":{"key2":"value2","key":"value"},"key2":"value2","key":"value"}]}
2.19.1
{"name":"name","items":[{"type":"type","key2":"value2","key":"value"}]}
Comment From: mikethecalamity
The 2.19.1 serialization is definitely what I would expect. So I think that was fixed. But in our system we receive the JSON from another service, so serialization was never an issue. The issue is definitely on the deserialization side of things.
The test case I put in this comment does a better job representing that, since it doesn't contain serialization, it just starts with the raw JSON. https://github.com/FasterXML/jackson-databind/issues/5205#issuecomment-3002412456
If you think this issue is getting too muddy, I can close it and open a new one starting fresh.
Comment From: pjfanning
@mikethecalamity can we keep this issue but can you change the test case so that the JSON text is provided in the test as opposed to relying on a writeValueAsString call?
Comment From: mikethecalamity
@pjfanning Now that I have more information with the help you've given me, I think my initial assessment of the issue is wrong. I renamed the issue and rewrote the description to more accurately represent the problem as I see it now. It doesn't seem to be an issue isolated to 2.19 (it happens in 2.17 and 2.18 too, from what I tested). The 2.19 version just revealed the error because it actually fixed the serialization to be as expected, thus causing the deserialization to now fail.
Comment From: janrieke
Just to let you know: I will also investigate this issue from lombok's side in the next days.
Comment From: janrieke
As already mentioned in projectlombok/lombok#3898: The Jackson docs state that @JsonAnySetter is a
marker annotation that can be used to define a logical "any setter" mutator -- either using non-static two-argument method (first argument name of property, second value to set) or a field (of type Map) or POJO
For builders, only methods can be used, but in your test case, there is no such method that has two arguments, only a one-argument method.
With lombok, use @JsonAnySetter @Singular and the test succeeds. Without lombok, change the builder setter method signature to @JsonAnySetter public ItemBuilder attributes(String key, Object value).
Comment From: mikethecalamity
Thanks