2

I have to store more than 100 millions of key-values in my HashMultiMap (key can have multiple values). Can anybody help me which one is faster for both storing and searching:

1) Berkeley DB

2) Tokyo Cabinet

3) H2

4) EhCache

5) Or anyothers

Another point, is performance of those approximately identical to in-memory hash map ? A little bit guidance will be more helpful. Thanks.

NB: information about any one of these is also helpful.

6
  • What size are your key/values? What structure do they have? Mar 30, 2012 at 14:10
  • @JeffFoster, key/values are int.
    – Arpssss
    Mar 30, 2012 at 14:11
  • @JeffFoster, more specifically stackoverflow.com/questions/9930347/multimap-space-issue-guava
    – Arpssss
    Mar 30, 2012 at 14:12
  • 1
    Are your ints randomly distributed over the 32bit space? I don't see why you couldn't store it all in memory (with a hand-written unboxed implementation of MultiMap). Alternatively you could try Redis. Mar 30, 2012 at 14:19
  • @JeffFoster, yes, distributed. I have to provide very limited memory to my JVM (2GB). If I want to store 100 millions ints, it will take more than 3 GB.
    – Arpssss
    Mar 30, 2012 at 14:27

2 Answers 2

3

I'd recommend Redis. It's more of a data structure store than the others (e.g. it supports maps and sets) and requires very little effort to get started with. It's incredibly simple to use and has a great Java API.

I'd start by looking at the functions here that allow you to associate a key with a group of values.

2
  • Thanks a lot. From your experence, is performance of Redis is approximately identical to in-memory hash map ?
    – Arpssss
    Mar 30, 2012 at 14:36
  • It'll depend on your circumstances. I've never had a problem with performance in Redis. Best advice would be to hack together a prototype and just see what happens. See also redis.io/topics/benchmarks Mar 30, 2012 at 14:37
2

I would have a look at leveldb, because it claims to be faster than Kyoto Cabinet. Other interesting libraries are Kyoto Cabinet, and JDBM3.

Those are low level libraries (without support for relational queries or SQL). Some offer transactional support, but I'm not sure if you need it. Still, in-memory hash maps are faster.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.