[JAVA-2648] POJOs with Collections not deserialized without a setter Created: 31/Oct/17  Updated: 29/Oct/23  Resolved: 04/Dec/17

Status: Closed
Project: Java Driver
Component/s: POJO
Affects Version/s: 3.5.0
Fix Version/s: 3.6.0

Type: New Feature Priority: Major - P3
Reporter: Paul Carter-Brown Assignee: Ross Lawley
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

JDK 8, Linux



 Description   

When using the new POJO codecs, a POJO with a collection that has no setter is serialised to Mongo successfully but when deserialised, the POJO does not have the collection set.

This is contrary to the behavior of most codecs such as GSON, Jackson, Johnzon etc etc which by default use the getter to get access to the collection and then write the entries. This issue causes POJOs created with JAXB from XSD not work with the Mongo BSON POJO Codec which is very inconvenient as all other frameworks work fine with them.

E.g a Person like this can be written and read from Mongo using the POJO codec provider without any data loss.

public class Person {
    private String name;
    private List<Pet> pets;
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public List<Pet> getPets() {
        return pets;
    }
    public void setPets(List<Pet> pets) {
        this.pets = pets;
    }
}
 
public class Pet {
    private String type;
    public String getType() {
        return type;
    }
    public void setType(String type) {
        this.type = type;
    }
}

But like this it writes and reads without an exception but the Person read does not have any pets in it:

public class Person {
    private String name;
    private List<Pet> pets;
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public List<Pet> getPets() {
        if (pets == null) {
            pets = new ArrayList<>();
        }
        return pets;
    }
}
 
public class Pet {
    private String type;
    public String getType() {
        return type;
    }
    public void setType(String type) {
        this.type = type;
    }
}

It would be great if the Codec could handle this by using the getter and populating the collection accordingly.

Thanks
Paul



 Comments   
Comment by Githook User [ 05/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added the USE_GETTERS_FOR_SETTERS convention

Allows the getters for Map / Collection properties to provide
the initial data, which is then mutated.

JAVA-2648
Branch: 3.6.x
https://github.com/mongodb/mongo-java-driver/commit/cf2a6a3cb5eb5d3f8b65ae119d6505d38248a3ee

Comment by Ross Lawley [ 05/Dec/17 ]

Hi paulcb,

I've also added a USE_GETTERS_FOR_SETTERS convention which mimics the JAXB behaviour. This must be manually added to the list of conventions when using the PojoCodecProvider.builder().register method.

We cannot add this convention to the existing default conventions at this time. The java driver follows semantic versioning and as the next release is only a minor version release, we cannot change the existing behaviour of the PojoCodec that users may be relying on.

All the best,

Ross

Comment by Githook User [ 05/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added the USE_GETTERS_FOR_SETTERS convention

Allows the getters for Map / Collection properties to provide
the initial data, which is then mutated.

JAVA-2648
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/cf2a6a3cb5eb5d3f8b65ae119d6505d38248a3ee

Comment by Paul Carter-Brown [ 05/Dec/17 ]

Hi Ross,

What is the downside of allowing the collection to be added to via the getter? I only see advantages and no disadvantages.

Comment by Ross Lawley [ 05/Dec/17 ]

Hi paulcb,

When testing against private fields with jackson, it used reflection and set the fields directly.

The PojoCodec takes inspiration from JSON-B (JSR 367) where the 3 constrains listed previously are enforced, this is because if follows the Java Bean Convention.

The SET_PRIVATE_FIELDS_CONVENTION allows users a way to mutate private fields, which is against Java bean convention. JAXB by using getters to set fields for collections or maps only, negates the apparently immutibility of these collections and forces mutation of the collection. This again seems against the Java bean convention that the PojoCodec follows.

So out of the box JAXB is not supported but I believe the SET_PRIVATE_FIELDS_CONVENTION should still fix the issue with JAXB interop?

Ross

Comment by Paul Carter-Brown [ 04/Dec/17 ]

Hi Ross,

Are you sure that Jackson and others use private field access via reflection?

See this:

https://fasterxml.github.io/jackson-databind/javadoc/2.8/com/fasterxml/jackson/databind/MapperFeature.html#USE_GETTERS_AS_SETTERS

public static final MapperFeature USE_GETTERS_AS_SETTERS
Feature that determines whether otherwise regular "getter" methods (but only ones that handle Collections and Maps, not getters of other type) can be used for purpose of getting a reference to a Collection and Map to modify the property, without requiring a setter method. This is similar to how JAXB framework sets Collections and Maps: no setter is involved, just setter.
Note that such getters-as-setters methods have lower precedence than setters, so they are only used if no setter is found for the Map/Collection property.

Feature is enabled by default.

From what I can see this works even if private access is disabled when deserializing.

My sense is that JAXB generated classes should just work in the Mongo mapper just like they do in all other mappers. I'd be hesitant to have to change a default property SET_PRIVATE_FIELDS_CONVENTION to true just get the default behavior of the "norm".

JAXB specifically wants collections to be accessed via the getter. This was a specific design choice and the Mongo driver would be sidestepping this.

Quote from JAXB spec:
"Design Note –
There is no setter method for a List property. The getter returns the List by reference. An item can be added to the List returned by the getter method using an appropriate method defined on
java.util.List
. Rationale for this design in JAXB 1.0 was to enable the implementation to wrapper the list and be able to perform checks as content was added or removed from the List."

Comment by Githook User [ 04/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added documentation for the SET_PRIVATE_FIELDS_CONVENTION

JAVA-2648
Branch: 3.6.x
https://github.com/mongodb/mongo-java-driver/commit/80b935e669f3dd6d9a0eb8b5d39a3b74928fd355

Comment by Ross Lawley [ 04/Dec/17 ]

Hi paulcb, I was verging on closing this ticket as "Won't Fix" for the following reasons.

The reason Jackson et al work is not because they mutate the collections, maps or fields, rather they use reflection and ignore the private nature of the field and set them anyway.

When creating the PojoCodec we made the decision to respect the user declared access modifiers for the given fields. So essentially, if a field is declared private and there is no setter (or annotated constructor) then the PojoCodec respects that decision and doesn't try to update it.

There are 3 quick fixes for the scenario where there is no setter but a user would like the PojoCodec to set the value:

  • Add a setter method
  • Declare the access to the field as public
  • Add a constructor and annotate with the @BsonCreator and @BsonProperty annotations.

However, I appreciate that changing existing POJOs that currently interact with JSON libraries just to work with MongoDB can be a cause of user pain. So in the 3.6.0 release there is a new SET_PRIVATE_FIELDS_CONVENTION convention, this will amend the PropertyAccessor for properties without a setter to use reflection to set the field directly. Conventions can be registered when creating the PojoCodecProvider.

I hope that will help!

Ross

Comment by Githook User [ 04/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added documentation for the SET_PRIVATE_FIELDS_CONVENTION

JAVA-2648
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/80b935e669f3dd6d9a0eb8b5d39a3b74928fd355

Comment by Githook User [ 04/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added a SET_PRIVATE_FIELDS_CONVENTION

Some existing JSON libraries set field values directly.

While we respect field access modifiers by default, this commit adds a
`SET_PRIVATE_FIELDS_CONVENTION` that can be used to directly set
private fields, thus mimicing other libraries.

This convention is not part of the default conventions and must be explicitly set.

JAVA-2648
Branch: 3.6.x
https://github.com/mongodb/mongo-java-driver/commit/3095be0853b0e2c047ecb123cdd0fb09b7a94901

Comment by Githook User [ 04/Dec/17 ]

Author:

{'username': 'rozza', 'email': 'ross.lawley@gmail.com', 'name': 'Ross Lawley'}

Message: Added a SET_PRIVATE_FIELDS_CONVENTION

Some existing JSON libraries set field values directly.

While we respect field access modifiers by default, this commit adds a
`SET_PRIVATE_FIELDS_CONVENTION` that can be used to directly set
private fields, thus mimicing other libraries.

This convention is not part of the default conventions and must be explicitly set.

JAVA-2648
Branch: master
https://github.com/mongodb/mongo-java-driver/commit/3095be0853b0e2c047ecb123cdd0fb09b7a94901

Comment by Nic Cottrell [ 01/Dec/17 ]

Thanks Ross. Is there a neat way for me to subclass PropertyMetadata and thus change the logic of isDeserializable() so that I can deserialize protected fields also? (I believe protected fields should be accessible by default via reflection).

Comment by Ross Lawley [ 01/Dec/17 ]

nicholas.cottrell by default we follow the the following convention for property getters and setters:

First the default PropertyAccessor will check to see if the POJO follows java bean convention, if so it will use the getter/setter methods to set properties. Alternatively, if there is a field associated with the Property and that field is public and not static/transient we will set the value directly. If the field is private, static or transient then it wont be set.

That said, you can create your own PropertyAccessor implementations to mutate the class as desired.

Comment by Nic Cottrell [ 30/Nov/17 ]

This applies to simple fields too. If setName() is removed from Person then that field can't be deserialized either. I believe this behaviour works fine in Morphia - maybe we can borrow logic from there?

Comment by Paul Carter-Brown [ 01/Nov/17 ]

Added pull request:
https://github.com/mongodb/mongo-java-driver/pull/424

Generated at Thu Feb 08 08:57:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.