13

I'm aware of DynamoDBMapper but in my case I can't use it because I don't know all the attributes beforehand.

I have a JSON and it's parsed to a map of objects by using Jackson parser:

Map<String, Object> userData = mapper.readValue(new File("user.json"), Map.class);

Looping through each attribute, how can I convert the value to AttributeValue given that DynamoDB AttributeValue supports Boolean, String, Number, Bytes, List, etc.

Is there an efficient way to do this? Is there a library for this already? My naive approach is to check if each value is of type Boolean/String/Number/etc. and then call the appropriate AttributeValue method, e.g: new AttributeValue().withN(value.toString()) - which gives me long lines of if, else if

3
  • I have a similar use case. How do you do the opposite - convert Map<String,AttributeValue> to an Item? I have streams enabled but the DynamoDBKinesisStreamsAdapter returns a Map<String,AttributeValue> ... see docs.aws.amazon.com/amazondynamodb/latest/developerguide/… .. I want to convert the AttributeValue to Object (Item returns a Map<String,Object> on .asMap)
    – shrewquest
    Mar 14, 2016 at 9:48
  • @shrewquest Did you have any luck in finding a solution? Apr 28, 2017 at 7:08
  • 1
    @shrewquest Item.fromMap( InternalUtils.toSimpleMapValue( mapOfAttributes ) ) Feb 22, 2018 at 22:51

6 Answers 6

45

Finally figured out by looking at how AWS parses the JSON

Basically, this is the code:

    Item item = new Item().withJSON("document", jsonStr);
    Map<String,AttributeValue> attributes = InternalUtils.toAttributeValues(item);
    return attributes.get("document").getM();

Very neat.

6
  • 12
    how I can convert from Map<String,AttributeValue> to Object ?
    – boden
    Jan 10, 2017 at 17:30
  • 2
    This is a fantastic find. Thank you.
    – saccharine
    Sep 14, 2017 at 23:50
  • 1
    @BohdanKorinnyi The approach taken by @BPm also has a method for converting the Map<String,AttributeValue> to List<Item>, which would allow you to then have access to the actual primitive types (e.g. like item.getBoolean(attrName)). If that fits your use-case, then see: toItemList
    – ecoe
    Apr 2, 2018 at 17:32
  • 1
    The InternalUtils class is deprecated (use ItemUtils instead). Apr 9, 2019 at 8:26
  • 1
    This method doesn't work if the source Object has a ByteBuffer field. Json converts it into struct and AttributeValue considers it to be a Map.
    – GP cyborg
    Jun 26, 2019 at 22:33
6

Following is a simple solution which can be applied to convert any DynamoDB Json to Simple JSON.

//passing the reponse.getItems() 
public static Object getJson(List<Map<String,AttributeValue>> mapList) {
    List<Object> finalJson= new ArrayList();
    for(Map<String,AttributeValue> eachEntry : mapList) {
        finalJson.add(mapToJson(eachEntry));
    }
    return finalJson;
}


//if the map is null then it add the key and value(string) in the finalKeyValueMap
public static Map<String,Object> mapToJson(Map<String,AttributeValue> keyValueMap){
    Map<String,Object> finalKeyValueMap = new HashMap();
    for(Map.Entry<String, AttributeValue> entry : keyValueMap.entrySet()) 
    {
        if(entry.getValue().getM() == null) {
            finalKeyValueMap.put(entry.getKey(),entry.getValue().getS());
        }
        else {
            finalKeyValueMap.put(entry.getKey(),mapToJson(entry.getValue().getM()));
        }
    }
    return finalKeyValueMap;
}

This will produce your desired Json in the form of List<Map<String,Object>> which is subset of the object.

2
  • this code only convert values which is of string type not other types like numbers etc.
    – Sunny Goel
    Apr 11 at 20:53
  • can you update your code to work for all types values ?
    – Sunny Goel
    Apr 11 at 20:54
6

I used JacksonConverterImpl to convert JsonNode to Map<String, AttributeValue>

ObjectMapper objectMapper = new ObjectMapper();
JsonNode jsonNode = objectMapper.readValue(jsonString, JsonNode.class);
final JacksonConverter converter = new JacksonConverterImpl();
Map<String, AttributeValue> map = converter.jsonObjectToMap(jsonNode);

Hope this helps!

Thanks, Jay

2

With the 2.x SDK, I have not yet identified an equivalent provided by the SDK. As an exercise I decided to solve this with switch expressions

This implementation will marshal maps, records, primitives, byte arrays, collection types and will call toString() on unknown types. The only datatype I did not implement was a byte set (BS) as it's not as trivial to determine unique byte array values.

You could add support for bean properties, but I figured it would be error prone and decided to just support toString in default.

Have included unmarshal as an exercise also.

import java.lang.reflect.InvocationTargetException;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Optional;
import java.util.Set;
import java.util.regex.Pattern;

import software.amazon.awssdk.core.SdkBytes;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import static java.util.stream.Collectors.toList;
import static java.util.stream.Collectors.toMap;
import static java.util.stream.Collectors.toSet;


public class AttributeValueConverter {

    public static AttributeValue fromCollection(Collection<? extends Object> c) {
        if (c instanceof Set s && s.stream().allMatch(n -> n instanceof String)) {
            var vals = c.stream()
                .map(n -> (String) n)
                .collect(toList());
            return AttributeValue.fromSs(vals);
        }
        var vals = c.stream()
            .map(x -> marshal(x))
            .collect(toList());
        return AttributeValue.fromL(vals);
    }

    public static AttributeValue fromMap(Map<? extends Object, ? extends Object> m) {
        var vals = new HashMap<String, AttributeValue>();
        m.forEach((k, v) ->
            vals.put(k.toString(), marshal(v))
        );
        return AttributeValue.fromM(vals);
    }

    public static AttributeValue fromRecord(Record r) {
        var vals = new HashMap<String, AttributeValue>();
        var components = r.getClass().getRecordComponents();
        for (var component : components) {
            try {
                vals.put(
                    component.getName(),
                    marshal(component.getAccessor().invoke(r))
                );
            } catch (IllegalAccessException | InvocationTargetException ex) {
                // will happen if the class is not accessible
            }
        }
        return AttributeValue.fromM(vals);
    }

    public static AttributeValue marshal(Object obj) {
        return switch (obj) {
            case null           -> AttributeValue.fromNul(true);
            case Optional opt   -> opt.isPresent()
                                   ? marshal(opt.get())
                                   : AttributeValue.fromNul(true);
            case Boolean b      -> AttributeValue.fromBool(b);
            case Number n       -> AttributeValue.fromN(n.toString());
            case String s       -> AttributeValue.fromS(s);
            // string representations of temporals are mostly what you want
            //case Temporal t     -> AttributeValue.fromS(t.toString());
            case Collection c   -> fromCollection(c);
            case Map m          -> fromMap(m);
            case Record r       -> fromRecord(r);
            case Object o when o.getClass().isArray() -> {
                var ctype = obj.getClass().getComponentType().getTypeName();
                if ("byte".equals(ctype)) {
                    yield AttributeValue.fromB(
                        SdkBytes.fromByteArray((byte[]) obj)
                    );
                }
                yield fromCollection(Arrays.asList((Object[]) obj));
            }
            default -> AttributeValue.fromS(obj.toString());
        };
    }

    private static final Pattern INT_PATTERN = Pattern.compile("^-?[0-9]+$");

    private static Number parseNumber(String n) {
        if (INT_PATTERN.matcher(n).matches()) {
            if (n.length() < 10) {
                return Integer.valueOf(n);
            }
            return Long.valueOf(n);
        }
        return Double.valueOf(n);
    }

    public static Object unmarshal(AttributeValue av) {
        return switch (av.type()) {
            case B      -> av.b().asByteArray();
            case BOOL   -> av.bool();
            case BS     -> av.bs().stream()
                            .map(SdkBytes::asByteArray)
                            .collect(toList());
            case L      -> av.l().stream()
                            .map(AttributeValueConverter::unmarshal)
                            .collect(toList());
            case M      -> av.m().entrySet().stream()
                            .collect(toMap(
                                Entry::getKey,
                                n -> Optional.ofNullable(unmarshal(n.getValue()))
                            ));
            case N      -> parseNumber(av.n());
            case NS     -> av.ns().stream()
                            .map(AttributeValueConverter::parseNumber)
                            .collect(toSet());
            case NUL    -> null;
            case S      -> av.s();
            case SS     -> Set.copyOf(av.ss());
            case UNKNOWN_TO_SDK_VERSION ->
                throw new UnsupportedOperationException("Type not supported");
        };
    }

}
1
  • 2
    Its absolutely insane there is not an equivalent for the 2.x SDK. Jan 25 at 1:43
0

For the 2.x SDK, use DynamoDBEnhancedClient.

DynamoDbClient ddb = DynamoDbClient.builder()
                .region(region)
                .credentialsProvider(credentialsProvider)
DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder()
     .dynamoDbClient(ddb)
     .build();
DynamoDbTable<Customer> custTable = enhancedClient.table("Customer", TableSchema.fromBean(Customer.class));
Customer record = new Customer();
            record.setCustName("Fred Pink");
            record.setId("id110");
            record.setEmail("[email protected]");
// Put the customer data into an Amazon DynamoDB table.
custTable.putItem(record);

Here is a full example: https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/javav2/example_code/dynamodb/src/main/java/com/example/dynamodb/enhanced/EnhancedPutItem.java

0

If you have an annotated bean (@DynamoDbBean) and just want to convert a POJO, I found this:

M bean = ....; TableSchema tableSchema = (TableSchema)TableSchema.fromClass(bean.getClass()); Map<String, software.amazon.awssdk.services.dynamodb.model.AttributeValue> itemMap = tableSchema.itemToMap(bean, true);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.