12

I'm trying to serialize an array of 7000 POJO using GSON and the serialization time is extremely slow. It's on the order of 3-5 seconds to serialize an array of the following object:

public class Case {
    private Long caseId;
    private Key<Organization> orgKey;

    private Key<Workflow> workflowKey;
    private Key<User> creatorKey;

    private Date creationTimestamp;
    private Date lastUpdatedTimestamp;

    private String name;
    private String stage;
    private String notes;
}

The key fields are serialized using a custom serializer/deserializer:

public class GsonKeySerializerDeserializer implements JsonSerializer<Key<?>>, JsonDeserializer<Key<?>>{

@Override
public JsonElement serialize(Key<?> src, Type typeOfSrc, JsonSerializationContext arg2) {
    return new JsonPrimitive(src.getString());
}

@Override
public Key<?> deserialize(JsonElement src, Type typeOfSrc, JsonDeserializationContext arg2) throws JsonParseException {
    if (src.isJsonNull() || src.getAsString().isEmpty()) {
        return null;
    }

    String s = src.getAsString();
    com.google.appengine.api.datastore.Key k = KeyFactory.stringToKey(s);
    return new Key(k);
}
}

To test the performance against hand writing a JSON serializer, I tested the following code and it could serialize the same array of Case objects approximately 10x faster than GSON.

List<Case> cases = (List<Case>) retVal;
JSONArray a = new JSONArray();
for (Case c : cases) {
    JSONObject o = new JSONObject();
    o.put("caseId", c.getCaseId());
    o.put("orgKey", c.getOrgKey().getString());
    o.put("workflowKey", c.getWorkflowKey().getString());
    o.put("creatorKey", c.getCreatorKey().getString());
    o.put("creationTimestamp", c.getCreationTimestamp().getTime());
    o.put("lastUpdatedTimestamp", c.getLastUpdatedTimestamp().getTime());
    o.put("name", c.getName());
    o.put("stage", c.getStage());
    o.put("notes", c.getNotes());
    a.put(o);

}
String json = a.toString();

Any ideas why GSON is performing so badly in this case?

UPDATE

Here's the code that actually starts the serialization:

Object retVal = someFunctionThatReturnsAList();
String json = g.toJson(retVal);
resp.getWriter().print(json);

UPDATE2

Here's a very simple test case that illustrates the poor performance relative to org.json:

List<Foo> list = new ArrayList<Foo>();
for (int i = 0; i < 7001; i++) {
    Foo f = new Foo();
    f.id = new Long(i);
    list.add(f);
}

Gson gs = new Gson();
long start = System.currentTimeMillis();
String s = gs.toJson(list);
System.out.println("Serialization time using Gson: " + ((double) (System.currentTimeMillis() - start) / 1000));


start = System.currentTimeMillis();
JSONArray a = new JSONArray();
for (Foo f : list) {
    JSONObject o = new JSONObject();
    o.put("id", f.id);
    a.put(o);

}
String json = a.toString();
System.out.println("Serialization time using org.json: " + ((double) (System.currentTimeMillis() - start) / 1000));

System.out.println(json.equals(s));

Where Foo is:

public class Foo {
public Long id;
}

This outputs:

Serialization time using Gson: 0.233
Serialization time using org.json: 0.028
true

An almost 10x performance difference!

6
  • Which gson version are you using and what's your code to produce the output?
    – Joey
    May 22, 2012 at 0:40
  • I'm using gson 2.2, updating the question with code to produce output
    – aloo
    May 22, 2012 at 1:10
  • How do you create your Gson object? May 22, 2012 at 14:11
  • GsonBuilder gson = new GsonBuilder(); gson.addSerializationExclusionStrategy(new GsonExclusionStrategy()); gson.registerTypeHierarchyAdapter(Key.class, new GsonKeySerializerDeserializer()); gson.registerTypeHierarchyAdapter(Date.class, new GsonDateSerializerDeserializer()); gson.registerTypeHierarchyAdapter(BlobKey.class, new GsonBlobKeySerializerDeserializer()); return gson.create();
    – aloo
    May 22, 2012 at 17:28
  • 1
    @aloo Yes GSON is taking a very long time to initialize, the serialization/deserialization is okay I guess, saves coding time, not runtime time, but initialization of GSON is very slow. Did you get a solution yet?
    – Pierre
    Mar 12, 2015 at 16:39

2 Answers 2

6

I attempted to reproduce your problem and could not. I created 7000 objects with nontrivial data in them. On my ThinkPad it took Gson ~260ms to serialize ~3MB of Gson, which is a respectable ~10Mbps.

A large fraction of that time was spent converting dates to strings. Converting the two date fields to 'long' saved about 50ms.

I was able to save another ~10ms by migrating from tree adapters (JsonSerializer/JsonDeserializer) to the new streaming adapter class TypeAdaper. The code that sets this up looks like this:

    private static TypeAdapter<Key<String>> keyAdapter = new TypeAdapter<Key<String>>() {
        @Override public void write(JsonWriter out, Key<String> value) throws IOException {
            out.value(value.value);
        }

        @Override public Key<String> read(JsonReader in) throws IOException {
            if (in.peek() == JsonToken.NULL) {
                in.nextNull();
                return null;
            }
            return new Key<String>(in.nextString());
        }
    };

    ...

    Gson gson = new GsonBuilder()
            .registerTypeAdapter(Key.class, keyAdapter)
            .create();

The main difference between my scenario and yours is that I'm using my own bogus Key class. But if the Key was the bottleneck that should have come up when you manually serialized each case.

Fixing the problem

Your best next step is to remove fields from Case until serialization improves. It's possible that one of your fields contains something that's taking a long time to serialize: perhaps a very long string that requires excessive escaping? Once you isolate the problem report a bug to the Gson project and we'll happily fix the problem. In addition to including the code that reproduces the problem you should also include representative data.

4
  • If it's excessive escaping is the issue wouldn't this show up in the manual serialization as well?
    – aloo
    May 22, 2012 at 15:53
  • Trying your other strategies now
    – aloo
    May 22, 2012 at 15:54
  • Ok I tried the TypeAdapter strategy and it doesn't seem to help that much. Whats interesting is that the version of my code that serializes "by hand", not only serializes the same fields (keys, dates, etc) using the same methodology as the TypeAdapters, but that it runs literally 20x faster. This seems to be something wrong with the gson library itself.
    – aloo
    May 22, 2012 at 18:52
  • I can replicate this bad performance with a VERY simple test. See the updated question.
    – aloo
    May 22, 2012 at 20:18
0

how about using flat buffers over json.

https://medium.freecodecamp.com/why-consider-flatbuffer-over-json-2e4aa8d4ed07#.d79exjq8n

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.