I need to generate table with say 600 consecutive numbers (starting with 51) in each row
How I do this with BigQuery Standard SQL?
2 Answers
Try GENERATE_ARRAY
in standard SQL:
SELECT num FROM UNNEST(GENERATE_ARRAY(51, 650)) AS num;
Edit: if you want more than about a million elements, you can use multiple calls to GENERATE_ARRAY
, although be warned that the query can end up taking a long time if you produce too many elements:
SELECT num1 * num2 AS num
FROM UNNEST(GENERATE_ARRAY(1, 1000000)) AS num1,
UNNEST(GENERATE_ARRAY(1, 100)) AS num2;
-
@Elliott Brossard, it looks like
GENERATE_ARRAY
is limiting the sequence length to exactly1048575
, which is2^20 - 1
. Do you know what that limit is for or where it comes from? My use case involves generating a sequence from 1 to 5 billion entries long. I'm working around it by writing the sequence to a bucket in CSV format, and importing to a table from there. Commented Jul 8, 2019 at 22:09 -
I added this limit with a comment that says,
Enforce a hard limit of around a million elements to avoid generating arrays that are too large for constant folding or other parts of the runtime.
I didn't want to make it easy to create arrays that would cause your query to fail. See my updated answer, though. Commented Jul 8, 2019 at 23:22
BigQuery Standard SQL
SELECT 50 + ROW_NUMBER() OVER() AS num
FROM UNNEST((SELECT SPLIT(FORMAT("%600s", ""),'') AS h FROM (SELECT NULL))) AS pos
ORDER BY num
BigQuery Legacy SQL
SELECT 50 + pos AS pos FROM (
SELECT ROW_NUMBER() OVER() AS pos, *
FROM (FLATTEN((SELECT SPLIT(RPAD('', 600, '.'),'') AS h FROM (SELECT NULL)), h))
) WHERE pos BETWEEN 1 AND 600
From there you can adjust logic for example to get consecutive days and other sequences
-
2I have something much nicer in the pipeline for standard SQL--unfortunately it isn't available yet :( I'll bookmark this post and add an answer once I have something to share. Commented Aug 10, 2016 at 23:09