2

I'm faced with a task to make our software stack scalable. It's currently not scalable because everything is lumped into a huge central Oracle database. Everyone accesses it, so it's always very busy, what's more, because of concerns of losing data, the database file is directly written onto netapp, so the disk access is slow.

We've had success with noSQL solutions with other tasks, so we are considering them. But one problem is, that current code relies heavily on Hibernate for its simplicity, because you can easily traverse a business object graph without worrying about loading the referenced objects.

For noSQL currently there is not such a Hibernate driver available; EVEN if there were, a problem with noSQL is that none of them supports JOIN, so that an efficient JOIN FETCH is impossible, and you would have to spend several trips to the store to fetch related objects. as a result, I'm inclined to think that noSQL is only good for projects with independent objects, instead of complex object graphs.

Any ideas?

10
  • 4
    Rather than move to NoSQL, have you looked into simply making your current RDBMS-centric systems simply work better? Eg. move the DB onto proper hardware with proper backup, shard out the DB, etc.
    – cdeszaq
    Jan 19, 2012 at 19:38
  • yes, 2nd-level cache has been used as much as possible, but since we have multiple application servers, write to cache won't be reflected in all servers, so tables with frequent updates can't use 2nd-level cache. EHCache has a "distributed" solution, but I have not seen a lot of adoption, so am not fully confident in it, plus it's not free. Jan 19, 2012 at 19:47
  • Those app-level fixes are good, but I was talking about fixing the database. Put the DB on good hardware (since it sounds like it isn't), and then consider sharding it out or breaking off pieces of it. You can also use memcache to cache things in the application layer more aggressively depending on your situation.
    – cdeszaq
    Jan 19, 2012 at 19:50
  • sharding probably won't help much, since our data volume is not big at all, it's just busy. (of course sharding would reduce the load per box , that may help), sharding will likely create the same problem of not being able to FETCH-JOIN same as noSQL, since you will now have to let oracle do a distributed JOIN. Jan 19, 2012 at 19:50
  • Perhaps sharding is the wrong term then, but the same idea applies if you break it out into master-slave replication. You have to know your workload (read heavy I would guess) and use that to tell you what you can do. Jumping to NoSQL when you are used to hibernate will be a much bigger shift than moving to slave-read, master-write.
    – cdeszaq
    Jan 19, 2012 at 19:53

2 Answers 2

2

I would recommend trying Infinispan as a distributed second level cache. This would allow you to cluster your application servers and still cache data in a read heavy application.

This guide is very useful for steps to get started with this. You'll also need to ensure that you're using JTA transaction management.

2

Another new thing to try is playOrm which has JQL/HQL BUT with some changes to make it compatible for noSql(ie. you need to partition 1 trillion row tables so you can join 1 partition with something else). It does all the ManyToOne, OneToMany, etc. etc.

There are other differences like a findAll method as is typical in noSql to grab 100 rows in one go as it is done in parallel to many servers.

An example scalable JQL query would be (notice the partition id as you would have your trades partitioned possibly by account or something that is meaningful to your domain).

@NoSqlQuery(name="findJoinOnNullPartition", query="PARTITIONS t(:partId) select t FROM TABLE as t INNER JOIN t.security as s where s.securityType = :type and t.numShares = :shares")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.