Infrastructure and preamble
I have a PlayFramework (2.3.8) App hosted on an AWS EC2 instance. I have an array of complex objects, which should be returned as a JSON string via a web API. I need a deep copy of the array, with all child objects fully loaded until the very last layer. The array has the size of 30-100 entries, each entry has around 1-10 entries, each entry of those has up to 100 properties, in the end there are no BLOBs or similar involved, it all boils down to strings/doubles/ints/bools. I am unsure how far the exact data structure is of importance, please let me know if you need more details. The resulting .json file size is about 1 MB.
The performance of deserializing this array is awful, for the ~1 MB on my local machine it takes 3-5 minutes; on the EC2 it takes about 20-30 seconds.
The initial problem: poor performance when using play.libs json
My array of objects is loaded and stored as a JsonNode. This JsonNode is then forwarded to an ObjectMapper, which finally writes it prettyPrinted:
List<myObject> myObjects = myObjectService.getInstance().getAllObjects(); // simplified example
JsonNode myJsonNode = Json.toJson(myObjects); // this line of code takes a huge amount of time!
ObjectMapper om = new ObjectMapper();
return om.writerWithDefaultPrettyPrinter().writeValueAsString(myJsonNode); // this runs in <10 ms
So I nailed down the culprit to be the Json.toJson deserialization. As far as I could find out, it is a sort-of-wrapped Jackson library which is used by the PlayFramework.
While I have read about some performance issues of JSON deserializing, I am unsure if we should be talking about some hundred-milliseconds to seconds, and not minutes. Anyway, I tried implementing some other JSON libraries (GSON, argonaut, flexjson), which didn't really go smoothly.
GSON
I "simply" tried replacing the play-json library with the GSON library, as I did on another small part of the project. It worked fine there, but even though I have NO circular references, it throws StackOverflowErrors at my face, even if I try to deserialize a tiny manually created object. Both on my dev machine as well as on the EC2 instance.
FlexJson
List<myObject> myObjects = myObjectService.getInstance().getAllObjects(); // simplified example
JSONSerializer serializer = new JSONSerializer().prettyPrint(true);
return serializer.deepSerialize(myObjects); // returns a prettyPrinted String
Worked quite okay so far, it takes only around 20% of the time compared to the Json.toJson method above. Which could be, however, because it doesn't REALLY deep copy the objects. It does deep copy it on the first layer, however since my model has some more complex properties (with childs and grandchilds and grandgrandchilds...), and quite a lot of them, I am unsure how to procede here.
Here is the example output of one of my nested objects (this is one of the properties of the "upper" object):
"class": "com.avaje.ebean.common.BeanList",
"empty": false,
"filterMany": null,
"finishedFetch": true,
"loaderIndex": 0,
"modifyAdditions": null,
"modifyListenMode": "NONE",
"modifyRemovals": null,
"populated": true,
"propertyName": "elements",
"readOnly": false,
"reference": false
Do you have any other solution suggestions, or hints what might be broken? I was also thinking about that maybe the entities are only FULLY loaded once I call .toJson()? Still it shouldn't take such an amount of time.
Thanks in advance!
.toJson()
". As I see, you are using Ebean. Try activate logging for sql statements to see if the object graph is being loaded before or during thetoJson
call.toJson
(if this process is quicker than the previous then clearly something other than json parsing/generation is making it slower)