40

We're storing organization names in a DynamoDB table on AWS, and would like to maintain official capitalization in those business names, for example in "TNT" and "FedEx".

Our use case is that users of the application can search for organizations by name, but we'd like that their queries are interpreted case-insensitively. So, queries for "FedEx", "Fedex" or "fedex" should all return the correct item in the table.

Other databases have ways to perform queries ignoring case (for example by the ILIKE key word in PostgreSQL), by expressing queries via regular expressions, or by applying functions in the condition (for example the LOWER() function).

How can this be done in DynamoDB? The documentation on Amazon DynamoDB's Query does not provide an answer.

(The best work-around seems to be storing the name twice: once with the official capitalization in effect, and once in another field with the name converted to lowercase. Searching should then be done on the latter field, with the query search term also converted to lowercase. Yes, I know it adds redundancy to the table. It's a work-around, not an optimal solution.)

4

2 Answers 2

43

yes, exactly, when you add the new item/row, add also a new field searchName, that is the lowercase (even more, maybe only letters/numbers/spaces) of the your name field. and then search by that searchName field

3
  • 21
    Yeah, DynamoDB sucks, lots of hacks and workaround for just simple things
    – Vedmant
    Jun 15, 2020 at 8:11
  • 20
    DynamoDB's whole purpose is to make it impossible to write a query that isn't scalable. That's why there are certain limitations and "workarounds" and why the data needs to be structured this way.
    – Andrew
    Nov 28, 2020 at 4:05
  • 4
    To @Andrew point, this is the age old software dilemma that everything is a tradeoff. DynamoDB is optimized for scalability at the cost of flexibility. If you think it sucks, then more likely it is a poor fit for your use case. Jun 15, 2022 at 18:00
-12

Writing duplicate data in dynamodb is not a good design. The best solution would be to add ' elastic search ' to dynamodb. You can connect this component ' out of the box' using the aws console. Then use custom anayzer in elastic search to get case insensitive data.

7
  • 8
    Instead of setting up Elasticsearch and making the integration work, wouldn't it be easier to create a secondary index (with all characters cast to lowercase) on the table, and query/scan that index as decribed on docs.aws.amazon.com/amazondynamodb/latest/developerguide/… ? Jun 1, 2017 at 8:08
  • 3
    A secondary index is probably an easier initial investment than setting up ElasticSearch, but in the long run it's less flexible and harder to maintain. The issue with a secondary index is that any time you update data in the search field, you also need to update the secondary index. If your data and/or schema change frequently, or if you have multiple pieces of code trying to write to the same table, all of them need to handle the secondary index consistently. ElasticSearch makes such indexing transparent. Aug 7, 2017 at 20:28
  • 33
    "Writing duplicate data in dynamodb is not a good design." For what it's worth I disagree with this statement as duplicate data is often a very good practice in a NoSQL database. It is often necessary to support your access patterns, or at least to monkey patch a database that didn't consider the access patterns enough from the start. Jan 2, 2020 at 17:20
  • 4
    Seconding the above responses. Setting up an entirely new storage type for such a simple operation (especially one as complex to manage as Elasticsearch) isn't justified. If the intent is to avoid duplicating data, it doesn't accomplish that either, as you still have the data in 2 places, except now it's easy for them to get out of sync. Adding an extra dynamo column is comparably simple and doesn't cause any issue. Apr 29, 2020 at 18:30
  • 1
    This answer is flat out wrong, DynamoDB often has duplicate data, such as in attributes which you need for your data access patterns, such as a sort key as sortKey : /${region}/${category}/${productName}, where all these attributes would also live in their own columns.
    – Derrops
    Feb 13, 2021 at 8:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.