2

We are developing an app which can perform search on products in our database. The database that we are using is Amazon Dynamo DB and there will be at least more than 10 million product information stored at any given point of time.

The user, through the UI enters a product name, and we want the search result to be displayed in less than 2-3 seconds. We understand that DynamoDB will play a major role in this search, but would like to know if there is anything else that can be done in the application side. We are planning to cache the search using Memcache, so that it will be faster during the second time search.

Is there anything else that we can either implement or take care of, so that the search happens really fast. The implementation is in Java.

Thanks in advance.

2
  • 2
    You need to be more specific about just what you mean by "search" - exact match, partial match, wildcard, simialr words, soundex, etc. Your DB will need to be indexed differently depending on the search types you wish to allow. Feb 11, 2013 at 9:55
  • @JoshGreifer - the search would can be both an exact match or partial match
    – JavaMany
    Feb 11, 2013 at 10:30

2 Answers 2

1

Dynamodb does not have the conecpt of indexes, so you won't be able to search fast by anything but the primary key.

You have some primary options:

  1. Pregenerate the search results and store the search key as a hash key and the id of the result as the range. Then you can get all search results by getting on the primary key
  2. Use another indexing to store all the id's. For instance CloudSearch
  3. If you want exact matches on product name, store them in a separate table with a mapping to the id
2
  • The exact fucntionality is the user searches for a product and all products matching within a radius of 5KM of his current locations are displayed. Any thoughts on this?
    – JavaMany
    Feb 11, 2013 at 11:11
  • @JavaMany That's a really really big scope for a question and hard to answer on SO. You will have to do calculations on the distance per item. Probably store them using town is a primary key, so you can get all items for his town and use a range key for the exact location (you can get range keys +- an integer) Feb 11, 2013 at 12:26
0

The exact functionality is the user searches for a product and all products matching within a radius of 5KM of his current locations are displayed. Any thoughts on this? – JavaMany

So, you want free-text search against product attributes, and geo-spatial filtering. DynamoDB doesn't support either of those two concepts "out of the box" (except by doing a full table scan), and doesn't really lend itself to relational queries.

You have some options - as others have mentioned, you could use CloudSearch for the free-text search attributes; that would filter the 10M records to a smaller subset; for each of those, you'd have to retrieve the long/lat from DynamoDB and compare them to a radius of 5KM around the user's point. Depending on the number of records CloudSearch finds, this may or may not perform acceptably.

You have to find a way to keep the CloudSearch index up to date.

The alternative is to use AmazonRDS, and use the free text search and geo-spatial features of MySQL or MS SQL. Of course, that may require a schema (which I assume you're trying to avoid).

Not the answer you're looking for? Browse other questions tagged or ask your own question.