Ordering problems with Sets in JSON

[deaddowney]

I'm running into a problem serializing my domain object to JSON

I have two classes

class Parent {

    static hasMany = [children: Child]
}


class Child {

    static belongsTo = [parent: Parent]

   static mapping = {
        sort 'id'
    }
   
}

I'm trying to serialize all of the children of a parent using the JSON converter.

def converter = parentInstance as JSON
String parentJson = converter.toString(false)

However the order of the children is all screwed up.  Looking at the code, I think the problem lies in DomainClassMarshaller.marshalObject
around line 127 in grails-plugin-converters-2.0.1

                        else if (referenceObject instanceof Set) {
                            referenceObject = new HashSet((Set) referenceObject);
                        }


The parent's reference to its children is a Hibernate PersistentSet class which implements Set.  Would it not be better if instead of returning a HashSet, the code returned a LinkedHashSet, preserving the order of the input set?

Is there any way, short of providing a different DomainClassMarshaller, to preserve the ordering in JSON?

Regards,
Adam

[deaddowney]

I've created a Jira for this issue:

http://jira.grails.org/browse/GRAILS-8849

[Ian Roberts]

On 22/02/2012 22:15, deaddowney wrote:
> Is there any way, short of providing a different DomainClassMarshaller, to
> preserve the ordering in JSON?

If you care about the ordering then shouldn't the association be
modelled as a List rather than a Set?

Ian

--
Ian Roberts               | Department of Computer Science
[hidden email]  | University of Sheffield, UK

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[deaddowney]

Ian,
  I am not explicitly stating in my code that I want a set.  All I'm specifying is a manyToOne relationship.  It's hibernate that's putting it into a PersistentSet.  Still, I don't see why Grails can't be nice and respect the original ordering when converting to JSON.   JSON, from what I understand doesn't distinguish between lists and sets, so I think the principle of least surprise would be to have an ordered Set, ala LinkedHashSet.

Regards,
Adam

[bobbywarner]

Specify List instead of Set on your mapped manyToOne.

http://grails.org/doc/latest/guide/GORM.html#sets,ListsAndMaps


Hope that helps!
Bobby

[Ian Roberts]

On 27/02/2012 17:58, deaddowney wrote:
> Ian,
>   I am not explicitly stating in my code that I want a set.  All I'm
> specifying is a manyToOne relationship.  It's hibernate that's putting it
> into a PersistentSet.  Still, I don't see why Grails can't be nice and
> respect the original ordering when converting to JSON.

Because there is no "original ordering" when the association is a Set.
Java Sets don't provide ordering guarantees, and if a Hibernate
PersistentSet preserves insertion order then that is IMHO an
implementation detail inside Hibernate (indeed, I wasn't aware that it
did so until I read this thread, though it's not really surprising when
you think about how you might implement a database-backed Set).  If you
need ordering guarantees then specify a collection type that provides
them, i.e. List (you control the order) or SortedSet (ordering
guaranteed based on Java compareTo).

One of the many places where there is an inevitable impedance mismatch
between the database view and the Java object graph view of the same
data model.

Ian

--
Ian Roberts               | Department of Computer Science
[hidden email]  | University of Sheffield, UK

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[deaddowney]

I agree that the Set interface provides no guarantees about the sorting order.  However there ARE no Sets in JSON, only Objects (Maps), Arrays (Collections), and Scalars
http://www.json.org/

For example, if I directly convert the Set to json:

parent.children as JSON

the conversion is correct because the CollectionMarshaller class is used, which just marshalls the collection in order.  There is no SetMarshaller, only a MapMarshaller (or similar) and a CollectionMarshaller.

The DomainClassMarshaller seems to be going out of its way to screw things up between Java and JSON:

I believe that all of this code:(excuse the formatting)
if (referenceObject instanceof SortedMap) {
  referenceObject = new TreeMap((SortedMap) referenceObject);
} else if (referenceObject instanceof SortedSet) {
  referenceObject = new TreeSet((SortedSet) referenceObject);
} else if (referenceObject instanceof Set) {
  referenceObject = new HashSet((Set) referenceObject);
} else if (referenceObject instanceof Map) {
  referenceObject = new HashMap((Map) referenceObject);
} else if (referenceObject instanceof Collection){
  referenceObject = new ArrayList((Collection) referenceObject);
}

could be replaced by this code:
if (referenceObject instanceof Map) {
  referenceObject = new LinkedHashMap((Map)referenceObject); }
else {
  referenceObject = new ArrayList((Collection) referenceObject);
}

which should be faster (not all those nasty instanceofs), more space efficient (List vs Set), drastically simpler, consistent with direct marshalling of the collection, and preserves the input order, which I believe is what the user wants.

[Ian Roberts]

On 28/02/2012 12:31, deaddowney wrote:

> I agree that the Set interface provides no guarantees about the sorting
> order.  However there ARE no Sets in JSON, only Objects (Maps), Arrays
> (Collections), and Scalars
> http://www.json.org/
>
> For example, if I directly convert the Set to json:
>
> *parent.children as JSON*
>
> the conversion is correct because the CollectionMarshaller class is used,
> which just marshalls the collection in order.

Any conversion that puts the right Child elements into the resulting
array (irrespective of order) is by definition "correct"...

Don't get me wrong, I think your patch is a good one in terms of
simplifying the logic and reducing the number of instanceof checks, but
there is nothing semantically "wrong" with the way it works now (except
maybe that it doesn't handle SortedSets that use a Comparator rather
than natural ordering, which I don't think is something that applies to
GORM domain class associations).

Ian

--
Ian Roberts               | Department of Computer Science
[hidden email]  | University of Sheffield, UK

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[deaddowney]

Ian Roberts wrote

Any conversion that puts the right Child elements into the resulting
array (irrespective of order) is by definition "correct"...

Don't get me wrong, I think your patch is a good one in terms of
simplifying the logic and reducing the number of instanceof checks, but
there is nothing semantically "wrong" with the way it works now (except
maybe that it doesn't handle SortedSets that use a Comparator rather
than natural ordering, which I don't think is something that applies to
GORM domain class associations).

We can argue about whether ordering factors into the definition of correctness, but I would argue that the output that the JSON converter is not what the user expects.  It is not consistent.  If you serialze at the root of a Domain class you get different results than if you serialize at the collection.  Why doesn't the CollectionMarshaller put a collection through the same gauntlet of instanceofs to ensure the same output as the DomainClassMarshaller?  Quite frankly, the fact that the marshaller is making shallow copies of all the collections is unexpected.

I don't mean to beat a dead horse with this discussion, but I appreciate your comments.

Thanks,
Adam