5

I'm working with two entities: Item and Attribute, which look something like the following:

Item
----
itemId

Attribute
---------
attributeId
name

An Item has Attributes, as specified in an association table:

ItemAttribute
--------------
itemId
attributeId

When this data gets to the client, it will be displayed with a row per Item, and each row will have a list of Attributes by name. For example:

Item  Attributes
----  ----------
1     A, B, C
2     A, C
3     A, B

The user will have the option to sort on the Attributes column, so we need the ability to sort the data as follows:

Item  Attributes
----  ----------
3     A, B
1     A, B, C
2     A, C

At present, we're getting one row of data per ItemAttribute row. Basically:

  SELECT Item.itemId,
         Attribute.name
    FROM Item
    JOIN ItemAttribute
      ON ItemAttribute.itemId = Item.itemId
    JOIN Attribute
      ON Attribute.attributeId = ItemAttribute.attributeId
ORDER BY Item.itemId;

Which produces a result like:

itemId  name
------  ----
1       A
1       B
1       C
2       A
2       C
3       A
3       B

The actual ORDER BY clause is based on user input. It's usually a single column, so the ordering is simple, and the app-side loop that processes the result set combines the Attribute names into a comma-separated list for presentation on the client. But when the user asks to sort on that list, it'd be nice to have Oracle sort the results so that -- using the example above -- we'd get:

itemId  name
------  ----
3       A
3       B
1       A
1       B
1       C
2       A
2       C

Oracle's LISTAGG function can be used to generate the attribute lists prior to sorting; however Attribute.name can be a very long string, and it is possible that the combined list is greater than 4000 characters, which would cause the query to fail.

Is there a clean, efficient way to sort the data in this manner using Oracle SQL (11gR2)?

8
  • Wouldn't adding ORDER BY Attributes to your query achieve what you are looking for?
    – Chandu
    Jul 18, 2012 at 12:55
  • @Chandu: I don't actually have Attributes as a string: that's simply how it will be displayed on the client after the comma-separated list is built by the application. I'd have to use something like LISTAGG to build that list within the query, but as soon as there is a list of more than 4000 characters (Oracle's maximum size for a VARCHAR2), the query will fail entirely.
    – Curtis F.
    Jul 18, 2012 at 13:11
  • Posting the query you are using now would help.
    – Chandu
    Jul 18, 2012 at 13:11
  • @Chandu Just updated the question with more information on the type of query being used at present.
    – Curtis F.
    Jul 18, 2012 at 13:20
  • @CurtisF. - so the sorting isn't really the issue, it's that listagg can only return 4000 chars, and you might exceed that; so you need something to replace listagg that isn't subject to that limit?
    – Alex Poole
    Jul 18, 2012 at 13:22

3 Answers 3

4

There are really two questions here:

1) How to aggregate more than 4000 characters of data

Is it even sensible to aggregate so much data and display it in a single column?

Anyway you will need some sort of large structure to display more than 4000 characters, like a CLOB for example. You could write your own aggregation method following the general guideline described in one of Tom Kyte's thread (obviously you would need to modify it so that the final output is a CLOB).

I will demonstrate a simpler method with a nested table and a custom function (works on 10g):

SQL> CREATE TYPE tab_varchar2 AS TABLE OF VARCHAR2(4000);
  2  /

Type created.

SQL> CREATE OR REPLACE FUNCTION concat_array(p tab_varchar2) RETURN CLOB IS
  2     l_result CLOB;
  3  BEGIN
  4     FOR cc IN (SELECT column_value FROM TABLE(p) ORDER BY column_value) LOOP
  5        l_result := l_result ||' '|| cc.column_value;
  6     END LOOP;
  7     return l_result;
  8  END;
  9  /

Function created.

SQL> SELECT item,
  2         concat_array(CAST (collect(attribute) AS tab_varchar2)) attributes
  3    FROM data
  4   GROUP BY item;

ITEM ATTRIBUTES
1    a b c
2    a c
3    a b

2) How to sort large data

Unfotunately you can't sort by an arbitrarily large column in Oracle: there are known limitations relative to the type and the length of the sort key.

  • Trying to sort with a clob will result in an ORA-00932: inconsistent datatypes: expected - got CLOB.
  • Trying to sort with a key larger than the database block size (if you decide to split your large data into many VARCHAR2 for example) will yield an ORA-06502: PL/SQL: numeric or value error: character string buffer too small

I suggest you sort by the first 4000 bytes of the attributes column:

SQL> SELECT * FROM (
  2     SELECT item,
  3            concat_array(CAST (collect(attribute) AS tab_varchar2)) attributes
  4       FROM data
  5      GROUP BY item
  6  ) order by dbms_lob.substr(attributes, 4000, 1);

ITEM ATTRIBUTES
3    a b
1    a b c
2    a c
2
  • Thanks! I doubt that the list will ever actually contain more than 4000 characters, but as that cannot be guaranteed, the query shouldn't bomb if it ever encounters such data. I agree that to display that much data doesn't necessarily make sense in the UI; so perhaps -- if it ever becomes a problem -- we'll go with an alternative means of display for lengthy lists.
    – Curtis F.
    Jul 18, 2012 at 14:38
  • I also was unaware of your second point (issues with sorting on large keys); given the small chance of it happening anyway, using only the first 4000 bytes is probably a great compromise. We'll have to see how this compares to app-side sorting in our particular solution. Either way, this technique of processing large lists might well prove useful.
    – Curtis F.
    Jul 18, 2012 at 14:43
1

As Vincent already said, sort keys are limited (no CLOB, max block size).

I can offer a slightly different solution which works out of the box in 10g and newer, without the need for a custom function and type using XMLAgg:

with ItemAttribute  as (
 select 'name'||level name
        ,mod(level,3) itemid
   from dual
  connect by level < 2000
)
,ItemAttributeGrouped as (
 select xmlagg(xmlparse(content name||' ' wellformed) order by name).getclobval() attributes
       ,itemid
   from ItemAttribute
  group by itemid
 )
select itemid
      ,attributes
      ,dbms_lob.substr(attributes,4000,1) sortkey
  from ItemAttributeGrouped
order by dbms_lob.substr(attributes,4000,1)
;  
0
0

Clean is subjective, and efficiency would need to be checked (but it's still only hitting the tables once so probably shouldn't be any worse), but if you have a finite upper limit on the number of attributes any item might - or at leads how many you have to consider for ordering - had then you could use multiple lead calls to do this:

SELECT itemId, name
  FROM (
    SELECT itemId, name, min(dr) over (partition by itemId) as dr
      FROM (
        SELECT itemId, name,
            dense_rank() over (order by name, name1, name2, name3, name4) as dr
          FROM (
            SELECT Item.itemId,
                     Attribute.name,
                     LEAD(Attribute.name, 1)
                         OVER (PARTITION BY Item.itemId
                             ORDER BY Attribute.name) AS name1,
                     LEAD(Attribute.name, 2)
                         OVER (PARTITION BY Item.itemId
                             ORDER BY Attribute.name) AS name2,
                     LEAD(Attribute.name, 3)
                         OVER (PARTITION BY Item.itemId
                             ORDER BY Attribute.name) AS name3,
                     LEAD(Attribute.name, 4)
                         OVER (PARTITION BY Item.itemId
                             ORDER BY Attribute.name) AS name4
                FROM Item
                JOIN ItemAttribute
                  ON ItemAttribute.itemId = Item.itemId
                JOIN Attribute
                  ON Attribute.attributeId = ItemAttribute.attributeId
               )
          )
      )
ORDER BY dr, name;

So, the inner query is getting the two values you care about, and using four lead calls (just as an example, so this can sort based on a maximum of the first five attribute names, but could of course be extended by adding more!) to get a picture of what else each item has. With your data this gives:

    ITEMID NAME       NAME1      NAME2      NAME3      NAME4
---------- ---------- ---------- ---------- ---------- ----------
         1 A          B          C
         1 B          C
         1 C
         2 A          C
         2 C
         3 A          B
         3 B

The next query out does a dense_rank over those five ordered attribute names, which assigns a rank to each itemID and name, giving:

    ITEMID NAME               DR
---------- ---------- ----------
         1 A                   1
         1 B                   4
         1 C                   6
         2 A                   3
         2 C                   6
         3 A                   2
         3 B                   5

The next query out finds the minimum of those calculated dr values for each itemId, using the analytic version of min, so each itemID=1 gets min(dr) = 1, itemId=2 gets 3, and itemId=3 gets 2. (You could combine those two levels by selecting min(dense_rank(...)) but that's (even) less clear).

The final outer query uses that minimum rank for each item to do the actual ordering, giving:

    ITEMID NAME
---------- ----------
         1 A
         1 B
         1 C
         3 A
         3 B
         2 A
         2 C
1
  • This is also a great solution to the problem. I wasn't familiar with Oracle's LEAD function -- but that might very well fit our needs here, as long as we can settle on an acceptable number of attributes that we'll actually use for sorting. Thanks!
    – Curtis F.
    Jul 18, 2012 at 14:53

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.