4

I would like to find the first 10 unused manual sequence numbers from a range.

Please find my query below:

select X1.* From
    (Select Rownum seq_number From Dual Connect By Rownum <= 
         (Select LPAD(9,(UTC.DATA_PRECISION - UTC.DATA_SCALE),9) 
          From User_Tab_Columns UTC 
          where UTC.Table_Name = 'Table_Name' And UTC.Column_Name = 'seq_number')) X1,
Table_Name X2
Where X1.seq_number = X2.seq_number (+)
  And X2.Rowid is Null
  And Rownum <= 10

Although it gives the required output I am worried about the load caused [if any] because we will be using this query multiple times a day.

Please advise if there is a way to optimize this query.

Additional Info: On the Table_Name T2, there is a unique index defined on (seq_number)

Work Example:

create table TEMP_TABLE_NAME ( seq_number number(6))

insert into TEMP_TABLE_NAME 
select distinct trunc(dbms_random.VALUE(1,5000)) seq_number 
from dual
connect by rownum <= 1000


create unique index TEMP_TABLE_NAME_IDX on TEMP_TABLE_NAME(seq_Number)


SELECT T1.*
  FROM (    SELECT ROWNUM seq_number
              FROM DUAL
        CONNECT BY ROWNUM <=
                      (SELECT LPAD (9,(UTC.DATA_PRECISION - UTC.DATA_SCALE),9)
                         FROM User_Tab_Columns UTC
                        WHERE     UTC.Table_Name = 'TEMP_TABLE_NAME'
                              AND UTC.Column_Name = 'SEQ_NUMBER')) T1,
       TEMP_TABLE_NAME T2
 WHERE     T1.seq_number = T2.seq_number(+)
       AND T2.ROWID IS NULL
       AND ROWNUM <= 10

For me my query gave the below output. Randomly created numbers in the table included 7 & 8 so they were ignored. The point is to get first 10 unused numbers.

1
2
3
4
5
6
9
10
11
12
6
  • 1
    Could you show sample data and expected result for this data ? I am trying to run your query on a test table with only 10 rows, but I am still getting ORA-30009: Not enough memory for CONNECT BY operation error, even for 10 rows.
    – krokodilko
    Jul 22, 2017 at 8:43
  • @krokodilko Hi, please check, i edited my post with a working example to replicate the exact scenario.
    – pOrinG
    Jul 22, 2017 at 8:53
  • Which version of Oracle are you on?
    – trincot
    Jul 22, 2017 at 9:10
  • @trincot Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
    – pOrinG
    Jul 22, 2017 at 9:26
  • How big is your table ? number(6) suggests that there is no more than 999999 rows, that is up to 1 million.
    – krokodilko
    Jul 22, 2017 at 9:39

1 Answer 1

2

First I would replace this sophisticated subquery:

Select Rownum seq_number From Dual Connect By Rownum <= 
         (Select LPAD(9,(UTC.DATA_PRECISION - UTC.DATA_SCALE),9) 
          From User_Tab_Columns UTC 
          where UTC.Table_Name = 'Table_Name' And UTC.Column_Name = 'seq_number')

with this one:

Select Rownum As seq_number From Dual 
Connect By Rownum <= (Select max( seq_number ) + 10 From TEMP_TABLE_NAME ) 

or even with a simple constant:

Select Rownum As seq_number From Dual Connect By Rownum <= 1000000

Your subquery frankly does not work for a very basic case:

create table TEMP_TABLE_NAME(
  seq_number NUMBER
);

SELECT LPAD (9,(UTC.DATA_PRECISION - UTC.DATA_SCALE),9) as x , 
       UTC.DATA_PRECISION, UTC.DATA_SCALE, UTC.COLUMN_NAME
FROM User_Tab_Columns UTC
WHERE     UTC.Table_Name = 'TEMP_TABLE_NAME'
  AND UTC.Column_Name = 'SEQ_NUMBER'
;

X        DATA_PRECISION DATA_SCALE COLUMN_NAME
-------- -------------- ---------- -----------
  (null)         (null)     (null) SEQ_NUMBER

And a second case:

create table TEMP_TABLE_NAME(
  seq_number NUMBER(15,0)
);

in this case the subquery tries to generate 999999999999999 rows, which quickly leads to out of memory error

SELECT count(*) FROM (
 SELECT ROWNUM seq_number
              FROM DUAL
        CONNECT BY ROWNUM <=
                      (SELECT LPAD (9,(UTC.DATA_PRECISION - UTC.DATA_SCALE),9)
                         FROM User_Tab_Columns UTC
                        WHERE     UTC.Table_Name = 'TEMP_TABLE_NAME'
                              AND UTC.Column_Name = 'SEQ_NUMBER')
);

ORA-30009: Not enough memory for CONNECT BY operation
30009. 0000 -  "Not enough memory for %s operation"
*Cause:    The memory size was not sufficient to process all the levels of the
           hierarchy specified by the query.
*Action:   In WORKAREA_SIZE_POLICY=AUTO mode, set PGA_AGGREGATE_TARGET to
           a reasonably larger value.
           Or, in WORKAREA_SIZE_POLICY=MANUAL mode, set SORT_AREA_SIZE to a
           reasonably larger value.

Secondly your query is not deterministic !!!
It strongly depends on a physical table structure and does not impose the correct order using ORDER BY clause.
Remember ->Wikipedia - ORDER BY

ORDER BY is the only way to sort the rows in the result set. Without this clause, the relational database system may return the rows in any order. If an ordering is required, the ORDER BY must be provided in the SELECT statement sent by the application.

Consider this test case:

create table TEMP_TABLE_NAME 
as SELECT * FROM (
    select rownum as seq_number , t.*
    from ALL_OBJECTS t
    cross join ( select * from dual connect by level <= 10)
    where rownum <= 100000
)
ORDER BY DBMS_RANDOM.Value;
create unique index TEMP_TABLE_NAME_IDX on TEMP_TABLE_NAME(seq_Number);

select count(*) from TEMP_TABLE_NAME;
  COUNT(*)
----------
    100000

DELETE FROM TEMP_TABLE_NAME
WHERE seq_number between 10000 and 10002
  OR seq_number between 20000 and 20002
  OR seq_number between 30000 and 30002
  OR seq_number between 40000 and 40002
  OR seq_number between 50000 and 50002
  OR seq_number between 60000 and 60002
  ;

If the index exists, then the result is OK:

SELECT T1.*
  FROM (    SELECT ROWNUM seq_number
              FROM DUAL
        CONNECT BY ROWNUM <= 1000000
) T1,
       TEMP_TABLE_NAME T2
 WHERE     T1.seq_number = T2.seq_number(+)
       AND T2.ROWID IS NULL
       AND ROWNUM <= 10
;

SEQ_NUMBER
----------
     10000
     10001
     10002
     20000
     20001
     20002
     30000
     30001
     30002
     40000

But what happens when some day someone deletes the index, or the optimizer for some reasons decides to not use that index ?
According to the definition: Without ORDER BY, the relational database system may return the rows in any order. I simulate these cases using a hint:

SELECT /*+ NO_INDEX(T2) */ T1.*
  FROM (    SELECT ROWNUM seq_number
              FROM DUAL
        CONNECT BY ROWNUM <= 1000000
) T1,
       TEMP_TABLE_NAME T2
 WHERE     T1.seq_number = T2.seq_number(+)
       AND T2.ROWID IS NULL
       AND ROWNUM <= 10
;

SEQ_NUMBER
----------
    213856
    910281
    668862
    412743
    295487
    214762
    788486
    346216
    777734
    806457

The below query enforces a proper order using ORDER BY clause and gives reproductibe results regardless of the proper index exists or not.
I am using the recommended ANSI SQL LEFT JOIN clause instead obsolete WHERE .... (+) syntax.

SELECT  * FROM (
    SELECT /*+ NO_INDEX(T2) */ T1.*
      FROM (    SELECT ROWNUM seq_number
                  FROM DUAL
            CONNECT BY ROWNUM <= 1000000
    ) T1 
    LEFT JOIN TEMP_TABLE_NAME T2
    ON T1.seq_number = T2.seq_number
    WHERE T2.ROWID IS NULL
    ORDER BY T1.seq_number
)
WHERE ROWNUM <= 10

Performance
The easiest way to check the performance is to do a test - run the query 10- 100 times and measure the time:

SET TIMING ON;
DECLARE
   x NUMBER;
BEGIN
   FOR i IN 1..10 LOOP
      SELECT sum( seq_number ) INTO x
      FROM (
           SELECT  * FROM (
            SELECT T1.*
              FROM (    SELECT ROWNUM seq_number
                          FROM DUAL
                    CONNECT BY ROWNUM <= 1000000
            ) T1 
            LEFT JOIN TEMP_TABLE_NAME T2
            ON T1.seq_number = T2.seq_number
            WHERE T2.ROWID IS NULL
            ORDER BY T1.seq_number
            )
            WHERE ROWNUM <= 10
        );
    END LOOP;
END;
/

PL/SQL procedure successfully completed.

Elapsed: 00:00:11.750

10 times - 11.75 sec, so one query takes 1,2 sec.


And a next version where a limit in CONNECT BY uses a subquery:

SET TIMING ON;
DECLARE
   x NUMBER;
BEGIN
   FOR i IN 1..10 LOOP
      SELECT sum( seq_number ) INTO x
      FROM (
           SELECT  * FROM (
            SELECT T1.*
              FROM (    SELECT ROWNUM seq_number
                          FROM DUAL
                    CONNECT BY ROWNUM <= (Select max( seq_number ) + 10 From TEMP_TABLE_NAME ) 
            ) T1 
            LEFT JOIN TEMP_TABLE_NAME T2
            ON T1.seq_number = T2.seq_number
            WHERE T2.ROWID IS NULL
            ORDER BY T1.seq_number
            )
            WHERE ROWNUM <= 10
        );
    END LOOP;
END;
/
PL/SQL procedure successfully completed.

Elapsed: 00:00:00.986

Much better - only 100 miliseconds.
This lead to the conclusion, that the CONNECT BY part is the most costly.


Another attempt that uses a table with pre-generated sequence of numbers up to 1 mln (kind of materialized view) instead of the CONNECT BY subquery that generates numbers each time on the fly in the memory:

create table seq(
   seq_number int primary key
)
ORGANIZATION INDEX ;

INSERT INTO seq 
SELECT level FROM dual
CONNECT BY LEVEL <= 1000000;

SET TIMING ON;
DECLARE
   x NUMBER;
BEGIN
   FOR i IN 1..10 LOOP
      SELECT sum( seq_number ) INTO x
      FROM (
           SELECT  * FROM (
            SELECT T1.*
            FROM seq T1 
            LEFT JOIN TEMP_TABLE_NAME T2
            ON T1.seq_number = T2.seq_number
            WHERE T2.ROWID IS NULL
            ORDER BY T1.seq_number
            )
            WHERE ROWNUM <= 10
        );
    END LOOP;
END;
/

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.398

This one is the fastest - only 40 ms

The first one 1200 ms, the last one 40ms - 30 times faster (3000 %).

1
  • Thanks for the answer & the tip on how to effectively optimize along with the workflow and examples. It is really helpful. Really appreciate it.
    – pOrinG
    Jul 22, 2017 at 23:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.