13

I have some entries with dates in my database. What is best?:

  • Fetch them with a sql statement and also apply order by.
  • Get the list with sql, and order them within the application with collection.sort or so?

Thanks

2
  • 3
    DB sorting is very fast. Highly improved. I doubt you can do better.
    – juergen d
    Commented Oct 28, 2012 at 10:58
  • @juergend Only if there's an index. If there isn't, the DB has no advantages at all in the sorting. However, sorting it in the DB might still be a win if it means you don't have to load as much data into the client in the first place. Commented Oct 29, 2012 at 7:28

4 Answers 4

10

This a very broad question that is very difficult to answer, and it depends a lot on what you mean by best?

  • From a performance perspective, you will simply have to measure to determine what part of your system is the bottleneck. Databases are usually very efficient, but it could still be relevant to off-load that work to the client.

  • From a separation of concern perspective, it depends on how the sorting matters in the application and how the application is layered.

Ask your self: "where does the knowledge that the data is sorted belong?" and "What would happen if I where to change from a relational database storage to something different".

4

To some extent, it depends on how many values are in the complete collection. If it is, say, 20-30 values then you can sort anywhere — even a relatively poor sorting algorithm can do that quickly (avoid Stooge Sort though; that's terrible) — as that is the sort of size of data chunk which you might expect to actually fetch in one service response.

But once you get into larger datasets you need to plan much more carefully. In particular, you want to avoid moving data around if you don't have to. If the data is currently only present in the database, you really don't want to fetch it all into the client just to sort it (a relatively expensive operation) and then throw virtually all of it away. It's far better to actually keep the data sorted in the database to start with, so that picking it up in order is trivial; in relational database terms, keeping the data sorted is functionally identical to maintaining an index on the data. Indeed, you can have multiple indices on the data, which can make even rather complex queries quick. (NoSQL DBs are more varied; some even don't support the concept of keeping data sorted.) The downside of maintaining indices is that they take up more space and they take time to maintain, particularly when the data is being created in the first place.

So… to return to your question, you probably want to try to not sort the data in the application: for most data, an appropriate index can be much more efficient as it lets your code not even look at unwanted data. But if you have to fetch it all into your application for some other reason and you can't bring it in pre-sorted, there's no reason to avoid sorting it yourself: Java's sorting algorithms are efficient and stable. But you should measure whether fetching it from the DB in the new order is faster. (The question is whether the DB overheads exceed the super-linear costs of re-sorting; lots of problems are in the domain where “maybe; hard to tell” is the answer.)

The other thing to balance is whether it is simpler for your code to not do sorting itself and instead always delegate that to the DB. Keeping your code simpler (and more bug-free) is a good goal to have…

0

Database management systems (DMBS) are optimized for these tasks, so I think you should stick with them. Especially if you are accessing the database from a script written in PHP or (other scripting language), it might be slower to perform that task using a script. You might also reach a memory limit allowed to be used by PHP if you sort the array using a script.

I don't mean to raise a question of performance of different programming languages, just want to point out that it is a very good practice to rely on the DMBS whenever you can.

2
  • 2
    The question is tagged with Java, so PHP isn't a question here. But I agree in general. The only situation where I would consider sorting in the program is when the sorting is done repeatedly by the user for a not-so-large-result. As that would require too DBMS roundtrips
    – user330315
    Commented Oct 28, 2012 at 11:03
  • Agree with @a_horse_with_no_name if it was say a UI grid/table with sorts and filters etc, client side would make sense as long as you get round the latency issue with a manual refresh, or a data has changed on the server notification. Commented Oct 28, 2012 at 11:21
-1

This is a very interesting question to me, and I want to present the other side of the accepted answer, which BTW is a very good answer with which I don't necessarily *dis*agree. Just want to present the other side. When I started in my career, I was working on mainframe DB2, and the old-timers that taught me were VERY INSISTENT that sorting be done OUTSIDE of the db. Their rational for this is that it's work that CAN be offloaded, and this leaves the DB free to service other requests. Of course, it's far more nuanced than this. In general, I'd say the factors you're weighing are: A) How busy, or central to your system, is your database? If your db is very busy, if you have a lot of OLTP processing on clients or app servers, and your client or application servers have lots of excess capacity, why not sort on the app server or client? Even if it's less efficient, it spreads the work through the system and gets you more throughput from a whole-systems perspective. B) How big is the sort? It would be silly to, say, blow your call stack or java heap because you sorted a gazillion MB of data. C) Will sorting in your app or app server cause pauses, latency, etc? In other words, if your particular programming language has REALLY bad sorting libraries, and you don't want to write your own, maybe letting the DB take 0.5 seconds is better than making your application take 5.0 seconds.

So, as with all things, "it depends" ;-). But, I think these are the things upon which it depends.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.