2

I have to develop a layer to retrieve data from a database (can be SQL Server, Oracle or IBM DB2). Queries (which are generic) are written by developers, but i can can modify them in my layer. The tables can be huge (say > 1 000 000 rows), and they are a lot of joins (for example, I have a query with 35 joins - no way to reduce).

So, I have to develop a pagination system, to retreive a "page" (say, 50 rows).

The layer (which is in a dll) is for desktop applications.

Important fact : queries are never ordered by ID.

The only way I found is to generate a unique row number (with MSSQL ROW_NUMBER() function) but won't work with Oracle because there are too much joins.

Does anyone know another way ?

2
  • at the end, do you want to do pagination in database or some middle layer?
    – HABJAN
    Mar 30, 2011 at 13:22
  • @HABJAN: We must see what is the fastest. In any case, my layer will return the "final" data to the application
    – Xavinou
    Mar 30, 2011 at 13:26

2 Answers 2

2

There are only two ways to do pagination code.

The first is database specific. Each of those databases have very different best practices with regards to paging through result sets. Which means that your layer is going to have to know what the underlying database is.

The second is to execute the query as is then just send the relevant records up the stream. This has obvious performance issues in that it would require your data layer to essentially grab all the records all of the time.


This is, IMHO, the primary reason why people shouldn't try to write database agnostic code. At the end of the day there are enough differences between RDBMs that it makes sense to have a pluggable data layer architecture which can take advantage of the specific RDBMs it works with.

In short, there is no ANSI standard for this. For example:

MySql uses the LIMIT keyword for paging.
Oracle has ROWNUM which has to be combined with subqueries. (not sure when it was introduced)
SQL Server 2008 has ROW_NUMBER which should be used with a CTE.
SQL Server 2005 had a different (and very complicated) way entirely of paging in a query which required several different procs and a function.
IBM DB2 has rownumber() which also must be implemented as a subquery.

2
  • 1
    AFAIK ROW_NUMBER() OVER was introduced in SQL Server 2005. SQL Server 2000 required you to jump through hoops. Anyway, if you expect most of the requests to access the first few pages only (like a list of search results), then you might consider a solution like this: Select top 100 * from (select top 10100 whatever_you_need from wherever_you_need order by id asc) order by id desc. This example gives records from 10000 to 10100. For large datasets it will be faster than an equivalent ROW_NUMER() OVER statement, because top actually stops the execution of the subquery after X records.
    – Vilx-
    Mar 30, 2011 at 15:01
  • select top 100 pp.* from (select top 100 your_columns from your_table order by ID asc) as pp order by pp.ID only way I managed to make vix's query worked Oct 5, 2012 at 15:12
0

You can do LINQ on your object collection, if you want to do that in the web side.

list.Skip(numPages * itemsPerPage).Take(itemsPerPage)

Lets you skip to the specified page (aka numPages = 0 is page 1).

2
  • That's whatever Enumerable you get from your database. My answer is valid only in the context that you get your results from the database and can enumerate over them somehow to implement your paging.
    – Tejs
    Mar 30, 2011 at 14:02
  • To use your method, I must retrieve all records from the database, which is impossible.
    – Xavinou
    Mar 30, 2011 at 14:32

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.