3

I've the following JSON:-

{
  "ROWNUM": "328938",
  "SOURCE_NAME": "I2323",
  "ID": "333333",
  "FIRST_NAME": "A121221",
  "KNOWN_AS": "G1223321",
  "LAST_NAME": "sadsadsd",
  "PLACE_OF_BIRTH": "Indsadsadsaddsaia",
  "DATE_OF_BIRTH": "sadsaddsa",
  "UPRN": "sadsadsad",
  "POST_CODE": "asdsadsda",
  "POST_TOWN": "GLASGOW",
  "ESTIMATED_DOB": "N",
  "LAST_UPDATED": "2019-02-11T13:57:05.264Z",
  "cluster_id": 3020,
  "aliases": [
    {
      "_id": {
        "timestamp": 1550152767,
        "machineIdentifier": 6505561,
        "processIdentifier": 59,
        "counter": 2775622,
        "time": 1550152767000,
        "timeSecond": 1550152767,
        "date": 1550152767000
      },
      "ROWNUM": "328938",
      "SOURCE_NAME": "I2323",
      "ID": "333333",
      "FIRST_NAME": "A121221",
      "KNOWN_AS": "G1223321",
      "LAST_NAME": "sadsadsd",
      "PLACE_OF_BIRTH": "Indsadsadsaddsaia",
      "DATE_OF_BIRTH": "sadsaddsa",
      "UPRN": "sadsadsad",
      "POST_CODE": "asdsadsda",
      "POST_TOWN": "GLASGOW",
      "ESTIMATED_DOB": "N",
      "LAST_UPDATED": "2019-02-11T13:57:05.264Z",
      "cluster_id": 3020,
      "score": "0.9997580647468567"
    },
    {
      "_id": {
        "timestamp": 1550152767,
        "machineIdentifier": 6505561,
        "processIdentifier": 59,
        "counter": 2775622,
        "time": 1550152767000,
        "timeSecond": 1550152767,
        "date": 1550152767000
      },
      "ROWNUM": "328938",
      "SOURCE_NAME": "I2323",
      "ID": "333333",
      "FIRST_NAME": "A121221",
      "KNOWN_AS": "G1223321",
      "LAST_NAME": "sadsadsd",
      "PLACE_OF_BIRTH": "Whatever",
      "DATE_OF_BIRTH": "sadsaddsa",
      "UPRN": "sadsadsad",
      "POST_CODE": "asdsadsda",
      "POST_TOWN": "PAISLEY",
      "ESTIMATED_DOB": "N",
      "LAST_UPDATED": "2019-02-11T13:57:05.264Z",
      "cluster_id": 3020,
      "score": "0.9997580647468567"
    }
  ]
}

Is there a jolt spec that would lowercase every key, including keys in nested objects? (in this case what is under aliases).

The following works for the top level keys but not for the nested ones:

[
  {
    // unwrap the keys and values into literal
    // "key" : "A", "value" : "b"
    "operation": "shift",
    "spec": {
      "*": {
        "$": "&1.key",
        "@": "&1.value"
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        // Now that the origional key
        //  is on the "right hand side"
        //  lowercase it
        "key": "=toLower"
      }
    }
  },
  {
    // pivot back, the now lowercased keys
    "operation": "shift",
    "spec": {
      "*": {
        "value": "@(1,key)"
      }
    }
  }
]

Thanks!

2
  • what about regex? If you type in a regex like this: ".*": then every key is matched
    – Noixes
    Commented Feb 17, 2019 at 10:00
  • I was hoping to keep things less brittle using a jolt transform, but if all hope is lost I might consider that! Commented Feb 17, 2019 at 10:12

2 Answers 2

6
+200

This produces the desired result extending the existing transformation:

[
  {
    // unwrap the keys and values into literal
    // "key" : "A", "value" : "b"
    "operation": "shift",
    "spec": {
      "*": {
        "$": "&1.key",
        "@": "&1.value"
      },
      //do the same for everything in aliases
      //&3 = aliases
      //&2 = array position
      //&1 = position of kvp
      "aliases": {
        "*": {
          "*": {
            "$": "&3.&2.&1.key",
            "@": "&3.&2.&1.value"
          }
        }
      }
    }
  },
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        // Now that the origional key
        //  is on the "right hand side"
        //  lowercase it
        "key": "=toLower"
      },
      "aliases": {
        "*": {
          "*": {
            // Now that the origional key
            //  is on the "right hand side"
            //  lowercase it
            "key": "=toLower"
          }
        }
      }
    }
  },
  {
    // pivot back, the now lowercased keys
    "operation": "shift",
    "spec": {
      "*": {
        "value": "@(1,key)"
      },
      "aliases": {
        "*": {
          "*": {
            //&3 = aliases
            //&2 = array postion
            //@(1,key) values from "key"
            "value": "&3.[&2].@(1,key)"
          }
        }
      }
    }
  }
]

Produces the following:

{
  "rownum": "328938",
  "source_name": "I2323",
  "id": "333333",
  "first_name": "A121221",
  "known_as": "G1223321",
  "last_name": "sadsadsd",
  "place_of_birth": "Indsadsadsaddsaia",
  "date_of_birth": "sadsaddsa",
  "uprn": "sadsadsad",
  "post_code": "asdsadsda",
  "post_town": "GLASGOW",
  "estimated_dob": "N",
  "last_updated": "2019-02-11T13:57:05.264Z",
  "cluster_id": 3020,
  "aliases": [
    {
      "_id": {
        "timestamp": 1550152767,
        "machineIdentifier": 6505561,
        "processIdentifier": 59,
        "counter": 2775622,
        "time": 1550152767000,
        "timeSecond": 1550152767,
        "date": 1550152767000
      },
      "rownum": "328938",
      "source_name": "I2323",
      "id": "333333",
      "first_name": "A121221",
      "known_as": "G1223321",
      "last_name": "sadsadsd",
      "place_of_birth": "Indsadsadsaddsaia",
      "date_of_birth": "sadsaddsa",
      "uprn": "sadsadsad",
      "post_code": "asdsadsda",
      "post_town": "GLASGOW",
      "estimated_dob": "N",
      "last_updated": "2019-02-11T13:57:05.264Z",
      "cluster_id": 3020,
      "score": "0.9997580647468567"
    },
    {
      "_id": {
        "timestamp": 1550152767,
        "machineIdentifier": 6505561,
        "processIdentifier": 59,
        "counter": 2775622,
        "time": 1550152767000,
        "timeSecond": 1550152767,
        "date": 1550152767000
      },
      "rownum": "328938",
      "source_name": "I2323",
      "id": "333333",
      "first_name": "A121221",
      "known_as": "G1223321",
      "last_name": "sadsadsd",
      "place_of_birth": "Whatever",
      "date_of_birth": "sadsaddsa",
      "uprn": "sadsadsad",
      "post_code": "asdsadsda",
      "post_town": "PAISLEY",
      "estimated_dob": "N",
      "last_updated": "2019-02-11T13:57:05.264Z",
      "cluster_id": 3020,
      "score": "0.9997580647468567"
    }
  ]
}
0
1

You can implement a custom JsonParserDelegate parser as demonstrated in the following test.

A working project can be check out from https://github.com/hovanessyan/json_lowercase_all_keys.git

a) create your own parser that lowercase the keys (KeysToLowercaseParser)

b) Override the JsonFactory and use the new Parser

I've pasted your json content in test.json, which is read in this test.

public class LowerCaseJsonTest {

    @Test
    public void name() throws IOException {

        ObjectMapper mapper = new ObjectMapper(new JsonFactory() {
            @Override
            protected JsonParser _createParser(byte[] data, int offset, int len, IOContext ctxt) throws IOException {
                return new KeysToLowercaseParser(super._createParser(data, offset, len, ctxt));
            }

            @Override
            protected JsonParser _createParser(InputStream in, IOContext ctxt) throws IOException {
                return new KeysToLowercaseParser(super._createParser(in, ctxt));
            }

            @Override
            protected JsonParser _createParser(Reader r, IOContext ctxt) throws IOException {
                return new KeysToLowercaseParser(super._createParser(r, ctxt));
            }

            @Override
            protected JsonParser _createParser(char[] data, int offset, int len, IOContext ctxt, boolean recyclable)
                    throws IOException {
                return new KeysToLowercaseParser(super._createParser(data, offset, len, ctxt, recyclable));
            }
        });


        File file = new File("src/main/resources/test.json");
        JsonNode jsonNode = mapper.readTree(file);
        String output = mapper.writeValueAsString(jsonNode);
        System.out.println(output);
    }

}

class KeysToLowercaseParser extends JsonParserDelegate {
    KeysToLowercaseParser(JsonParser d) {
        super(d);
    }

    @Override
    public String getCurrentName() throws IOException {
        if (hasTokenId(JsonTokenId.ID_FIELD_NAME)) {
            return delegate.getCurrentName().toLowerCase();
        }
        return delegate.getCurrentName();
    }

    @Override
    public String getText() throws IOException {
        if (hasTokenId(JsonTokenId.ID_FIELD_NAME)) {
            return delegate.getText().toLowerCase();
        }
        return delegate.getText();
    }
}
3
  • Hey, I was really looking for a Jolt spec since I'm wanting to integrate this into an Apache Nifi pipeline (with a Jolt processor), but I'll give you the rep just for the effort you put into that answer. Thanks! (Edit: says I can only award the bounty in 15 hours but will do.) Commented Feb 17, 2019 at 18:43
  • Hey, thanks but I didn't realize you were looking for a Jolt specific answer. Why don't you wait the whole bounty period - maybe someone will answer with a Jolt based solution. Commented Feb 17, 2019 at 18:47
  • Ok I will do, I should have been more clear in my question. If it all else fails I'll look at wrapping your code with an ExecuteScript processor in Nifi perhaps. Thanks again! Commented Feb 17, 2019 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.