I tried using random.randint(0, 100)
, but some numbers were the same. Is there a method/module to create a list unique random numbers?
-
2If they are unique they can be truly random in the right context. Like a random sample of indexes without replacement can still be completely random.– gbtimmonSep 24, 2016 at 19:20
21 Answers
This will return a list of 10 numbers selected from the range 0 to 99, without duplicates.
import random
random.sample(range(100), 10)
-
3This technique wastes memory, especially for large samples. I posted code for a much more memory and compute efficient solution below that uses a Linear Congruential Generator. Nov 30, 2018 at 4:56
-
It was pointed out to me that the LCG method is less "random" though, so if you want to generate many unique random sequences, the variety will be less than this solution. If you only need a handful of random sequences, LCG is the way to go! Nov 30, 2018 at 16:25
-
4
numpy
instead ofrandom
seems faster.import numpy as np; np.random.permutation(100)[:10]
also generates 10 numbers selected from 0 to 99, without duplicates. Benchmarking in IPython, yields 103 µs ± 513 ns for%timeit random.sample(range(1000), 100)
, and 17 µs ± 1.24 µs for%timeit np.random.permutation(1000)[:100]
. Sep 4, 2020 at 10:26 -
Adding to comment by @AntPlante, additionally use
.tolist()
if the output should be a list and not a numpy array. Feb 2, 2022 at 4:20 -
1@wjandrea yeah I'm aware that Python 3
range
produces a generator. Back when I posted that comment if you triedsample = random.sample(range(1000000000000000000), 10)
you could watch the memory of the process grow as it tried to materialize the range before extracting a sample. Checking now with Python 3.10 that appears to have been implemented differently (no memory issues), so my earlier comment is irrelevant now. The LCG solution is still a fun learning exercise though! 🤓 Dec 19, 2022 at 18:32
You can use the shuffle function from the random module like this:
import random
nums = list(range(1, 100)) # list of integers from 1 to 99
# adjust this boundaries to fit your needs
random.shuffle(nums)
print(nums) # <- List of unique random numbers
Note here that the shuffle method doesn't return any list as one may expect, it only shuffle the list passed by reference.
-
1It is good to mention here that xrange works only in Python 2 and not in Python 3. Feb 26, 2020 at 15:02
-
@ShayanShafiq: That said, on Python 3,
range
isxrange
(but slightly better), so just dropping thex
makes it work the same there (if you add the parensprint
requires too). Jun 3, 2022 at 18:45
You can first create a list of numbers from a
to b
, where a
and b
are respectively the smallest and greatest numbers in your list, then shuffle it with Fisher-Yates algorithm or using the Python's random.shuffle
method.
-
1Generating a full list of indices is a waste of memory, especially for large samples. I posted code for a much more memory and compute efficient solution below that uses a Linear Congruential Generator. Nov 30, 2018 at 4:57
-
Linear Congruential Pseudo-random Number Generator
O(1) Memory
O(k) Operations
This problem can be solved with a simple Linear Congruential Generator. This requires constant memory overhead (8 integers) and at most 2*(sequence length) computations.
All other solutions use more memory and more compute! If you only need a few random sequences, this method will be significantly cheaper. For ranges of size N
, if you want to generate on the order of N
unique k
-sequences or more, I recommend the accepted solution using the builtin methods random.sample(range(N),k)
as this has been optimized in python for speed.
Code
# Return a randomized "range" using a Linear Congruential Generator
# to produce the number sequence. Parameters are the same as for
# python builtin "range".
# Memory -- storage for 8 integers, regardless of parameters.
# Compute -- at most 2*"maximum" steps required to generate sequence.
#
def random_range(start, stop=None, step=None):
import random, math
# Set a default values the same way "range" does.
if (stop == None): start, stop = 0, start
if (step == None): step = 1
# Use a mapping to convert a standard range into the desired range.
mapping = lambda i: (i*step) + start
# Compute the number of numbers in this range.
maximum = (stop - start) // step
# Seed range with a random integer.
value = random.randint(0,maximum)
#
# Construct an offset, multiplier, and modulus for a linear
# congruential generator. These generators are cyclic and
# non-repeating when they maintain the properties:
#
# 1) "modulus" and "offset" are relatively prime.
# 2) ["multiplier" - 1] is divisible by all prime factors of "modulus".
# 3) ["multiplier" - 1] is divisible by 4 if "modulus" is divisible by 4.
#
offset = random.randint(0,maximum) * 2 + 1 # Pick a random odd-valued offset.
multiplier = 4*(maximum//4) + 1 # Pick a multiplier 1 greater than a multiple of 4.
modulus = int(2**math.ceil(math.log2(maximum))) # Pick a modulus just big enough to generate all numbers (power of 2).
# Track how many random numbers have been returned.
found = 0
while found < maximum:
# If this is a valid value, yield it in generator fashion.
if value < maximum:
found += 1
yield mapping(value)
# Calculate the next value in the sequence.
value = (value*multiplier + offset) % modulus
Usage
The usage of this function "random_range" is the same as for any generator (like "range"). An example:
# Show off random range.
print()
for v in range(3,6):
v = 2**v
l = list(random_range(v))
print("Need",v,"found",len(set(l)),"(min,max)",(min(l),max(l)))
print("",l)
print()
Sample Results
Required 8 cycles to generate a sequence of 8 values.
Need 8 found 8 (min,max) (0, 7)
[1, 0, 7, 6, 5, 4, 3, 2]
Required 16 cycles to generate a sequence of 9 values.
Need 9 found 9 (min,max) (0, 8)
[3, 5, 8, 7, 2, 6, 0, 1, 4]
Required 16 cycles to generate a sequence of 16 values.
Need 16 found 16 (min,max) (0, 15)
[5, 14, 11, 8, 3, 2, 13, 1, 0, 6, 9, 4, 7, 12, 10, 15]
Required 32 cycles to generate a sequence of 17 values.
Need 17 found 17 (min,max) (0, 16)
[12, 6, 16, 15, 10, 3, 14, 5, 11, 13, 0, 1, 4, 8, 7, 2, ...]
Required 32 cycles to generate a sequence of 32 values.
Need 32 found 32 (min,max) (0, 31)
[19, 15, 1, 6, 10, 7, 0, 28, 23, 24, 31, 17, 22, 20, 9, ...]
Required 64 cycles to generate a sequence of 33 values.
Need 33 found 33 (min,max) (0, 32)
[11, 13, 0, 8, 2, 9, 27, 6, 29, 16, 15, 10, 3, 14, 5, 24, ...]
-
1This is very cool! But I'm nut sure that it really answers the question; say I want to sample 2 values from 0 to 4. Without generating my own
prime
, the function will only return me 4 possible answers, becausevalue
is the only randomly chosen thing with 4 possible values, when we need at least (4 choose 2) = 6, (allowing for non-random ordering).random_range(2,4)
will return values {(1, 0), (3, 2), (2, 1), (0, 3)}, but never the pair (3,1) (or (1,3)). Are you expecting new randomly generated large primes each function call?– user8105524Nov 30, 2018 at 14:54 -
1(Also I'm assuming that you expect people to shuffle the sequence after your function returns it if they want random ordering, since
random_range(v)
returns up tov
unique sequences instead ofv!
)– user8105524Nov 30, 2018 at 15:15 -
Totally true! It's hard to balance between avoiding integer overflow and generating enough random sequences. I updated the function to incorporate a little more randomness, but it is still not as random as v!. It depends on if you want to use the function multiple times. This solution is best used when you are generating from a large range of values (when the memory consumption of others would be much higher). I'll think on it more, thanks! Nov 30, 2018 at 16:21
-
This algorithm is awesome. Exactly what I needed in a few different places in my project. On my computer it seems to outperform rand.randint too! Apr 13, 2022 at 16:47
-
1@KristofferLindvall yes, either do
np.asarray(list(random_range(...)))
ornumbers = np.zeros(N) ; for i,n in enumerate(random_range(N)): numbers[i] = n
. Aug 25, 2022 at 0:10
The solution presented in this answer works, but it could become problematic with memory if the sample size is small, but the population is huge (e.g. random.sample(insanelyLargeNumber, 10)
).
To fix that, I would go with this:
answer = set()
sampleSize = 10
answerSize = 0
while answerSize < sampleSize:
r = random.randint(0,100)
if r not in answer:
answerSize += 1
answer.add(r)
# answer now contains 10 unique, random integers from 0.. 100
-
Now
random.sample
uses this approach for small number of samples from a large population, so this problem with memory doesn't really exist anymore. Although, at the time this answer was written, the implementation ofrandom.shuffle
may have been different.– kyrillApr 26, 2020 at 16:22
If you need to sample extremely large numbers, you cannot use range
random.sample(range(10000000000000000000000000000000), 10)
because it throws:
OverflowError: Python int too large to convert to C ssize_t
Also, if random.sample
cannot produce the number of items you want due to the range being too small
random.sample(range(2), 1000)
it throws:
ValueError: Sample larger than population
This function resolves both problems:
import random
def random_sample(count, start, stop, step=1):
def gen_random():
while True:
yield random.randrange(start, stop, step)
def gen_n_unique(source, n):
seen = set()
seenadd = seen.add
for i in (i for i in source() if i not in seen and not seenadd(i)):
yield i
if len(seen) == n:
break
return [i for i in gen_n_unique(gen_random,
min(count, int(abs(stop - start) / abs(step))))]
Usage with extremely large numbers:
print('\n'.join(map(str, random_sample(10, 2, 10000000000000000000000000000000))))
Sample result:
7822019936001013053229712669368
6289033704329783896566642145909
2473484300603494430244265004275
5842266362922067540967510912174
6775107889200427514968714189847
9674137095837778645652621150351
9969632214348349234653730196586
1397846105816635294077965449171
3911263633583030536971422042360
9864578596169364050929858013943
Usage where the range is smaller than the number of requested items:
print(', '.join(map(str, random_sample(100000, 0, 3))))
Sample result:
2, 0, 1
It also works with with negative ranges and steps:
print(', '.join(map(str, random_sample(10, 10, -10, -2))))
print(', '.join(map(str, random_sample(10, 5, -5, -2))))
Sample results:
2, -8, 6, -2, -4, 0, 4, 10, -6, 8
-3, 1, 5, -1, 3
-
what if you are generate over 8 billion numbers, sooner or later seen will become too big Mar 12, 2018 at 23:00
-
This answer has a severe flaw for large samples. The probability of collision grows linearly with each step. I posted a solution using a Linear Congruential Generator that has O(1) memory overhead and O(k) steps required for generating k numbers. This can be solved much more efficiently! Nov 30, 2018 at 4:54
-
"This function resolves both problems" How does it resolve the second problem? You still can't take 1000 samples from a population of 2. Instead of throwing an exception you produce an incorrect result; that's hardly a resolution of the "problem" (which really isn't a problem to begin with since it's not at all reasonable to request k unique samples from a population of n < k).– kyrillApr 26, 2020 at 14:44
If the list of N numbers from 1 to N is randomly generated, then yes, there is a possibility that some numbers may be repeated.
If you want a list of numbers from 1 to N in a random order, fill an array with integers from 1 to N, and then use a Fisher-Yates shuffle or Python's random.shuffle()
.
Here is a very small function I made, hope this helps!
import random
numbers = list(range(0, 100))
random.shuffle(numbers)
-
How is that generate random numbers? The numbers will always be 0-99 Nov 26, 2022 at 17:33
A very simple function that also solves your problem
from random import randint
data = []
def unique_rand(inicial, limit, total):
data = []
i = 0
while i < total:
number = randint(inicial, limit)
if number not in data:
data.append(number)
i += 1
return data
data = unique_rand(1, 60, 6)
print(data)
"""
prints something like
[34, 45, 2, 36, 25, 32]
"""
One straightforward alternative is to use np.random.choice() as shown below
np.random.choice(range(10), size=3, replace=False)
This results in three integer numbers that are different from each other. e.g., [1, 3, 5], [2, 5, 1]...
The answer provided here works very well with respect to time as well as memory but a bit more complicated as it uses advanced python constructs such as yield. The simpler answer works well in practice but, the issue with that answer is that it may generate many spurious integers before actually constructing the required set. Try it out with populationSize = 1000, sampleSize = 999. In theory, there is a chance that it doesn't terminate.
The answer below addresses both issues, as it is deterministic and somewhat efficient though currently not as efficient as the other two.
def randomSample(populationSize, sampleSize):
populationStr = str(populationSize)
dTree, samples = {}, []
for i in range(sampleSize):
val, dTree = getElem(populationStr, dTree, '')
samples.append(int(val))
return samples, dTree
where the functions getElem, percolateUp are as defined below
import random
def getElem(populationStr, dTree, key):
msd = int(populationStr[0])
if not key in dTree.keys():
dTree[key] = range(msd + 1)
idx = random.randint(0, len(dTree[key]) - 1)
key = key + str(dTree[key][idx])
if len(populationStr) == 1:
dTree[key[:-1]].pop(idx)
return key, (percolateUp(dTree, key[:-1]))
newPopulation = populationStr[1:]
if int(key[-1]) != msd:
newPopulation = str(10**(len(newPopulation)) - 1)
return getElem(newPopulation, dTree, key)
def percolateUp(dTree, key):
while (dTree[key] == []):
dTree[key[:-1]].remove( int(key[-1]) )
key = key[:-1]
return dTree
Finally, the timing on average was about 15ms for a large value of n as shown below,
In [3]: n = 10000000000000000000000000000000
In [4]: %time l,t = randomSample(n, 5)
Wall time: 15 ms
In [5]: l
Out[5]:
[10000000000000000000000000000000L,
5731058186417515132221063394952L,
85813091721736310254927217189L,
6349042316505875821781301073204L,
2356846126709988590164624736328L]
-
You think that answer is complicated? What is this then?! And then there's the other answer, which generates many "spurious integers". I ran your implementation with example input you gave (populationSize = 1000, sampleSize = 999). Your version calls the
random.randint
function 3996 times, whereas the other one cca. 6000 times. Not that big of an improvement huh?– kyrillApr 26, 2020 at 17:10 -
In order to obtain a program that generates a list of random values without duplicates that is deterministic, efficient and built with basic programming constructs consider the function extractSamples
defined below,
def extractSamples(populationSize, sampleSize, intervalLst) :
import random
if (sampleSize > populationSize) :
raise ValueError("sampleSize = "+str(sampleSize) +" > populationSize (= " + str(populationSize) + ")")
samples = []
while (len(samples) < sampleSize) :
i = random.randint(0, (len(intervalLst)-1))
(a,b) = intervalLst[i]
sample = random.randint(a,b)
if (a==b) :
intervalLst.pop(i)
elif (a == sample) : # shorten beginning of interval
intervalLst[i] = (sample+1, b)
elif ( sample == b) : # shorten interval end
intervalLst[i] = (a, sample - 1)
else :
intervalLst[i] = (a, sample - 1)
intervalLst.append((sample+1, b))
samples.append(sample)
return samples
The basic idea is to keep track of intervals intervalLst
for possible values from which to select our required elements from. This is deterministic in the sense that we are guaranteed to generate a sample within a fixed number of steps (solely dependent on populationSize
and sampleSize
).
To use the above function to generate our required list,
In [3]: populationSize, sampleSize = 10**17, 10**5
In [4]: %time lst1 = extractSamples(populationSize, sampleSize, [(0, populationSize-1)])
CPU times: user 289 ms, sys: 9.96 ms, total: 299 ms
Wall time: 293 ms
We may also compare with an earlier solution (for a lower value of populationSize)
In [5]: populationSize, sampleSize = 10**8, 10**5
In [6]: %time lst = random.sample(range(populationSize), sampleSize)
CPU times: user 1.89 s, sys: 299 ms, total: 2.19 s
Wall time: 2.18 s
In [7]: %time lst1 = extractSamples(populationSize, sampleSize, [(0, populationSize-1)])
CPU times: user 449 ms, sys: 8.92 ms, total: 458 ms
Wall time: 442 ms
Note that I reduced populationSize
value as it produces Memory Error for higher values when using the random.sample
solution (also mentioned in previous answers here and here). For above values, we can also observe that extractSamples
outperforms the random.sample
approach.
P.S. : Though the core approach is similar to my earlier answer, there are substantial modifications in implementation as well as approach alongwith improvement in clarity.
The problem with the set based approaches ("if random value in return values, try again") is that their runtime is undetermined due to collisions (which require another "try again" iteration), especially when a large amount of random values are returned from the range.
An alternative that isn't prone to this non-deterministic runtime is the following:
import bisect
import random
def fast_sample(low, high, num):
""" Samples :param num: integer numbers in range of
[:param low:, :param high:) without replacement
by maintaining a list of ranges of values that
are permitted.
This list of ranges is used to map a random number
of a contiguous a range (`r_n`) to a permissible
number `r` (from `ranges`).
"""
ranges = [high]
high_ = high - 1
while len(ranges) - 1 < num:
# generate a random number from an ever decreasing
# contiguous range (which we'll map to the true
# random number).
# consider an example with low=0, high=10,
# part way through this loop with:
#
# ranges = [0, 2, 3, 7, 9, 10]
#
# r_n :-> r
# 0 :-> 1
# 1 :-> 4
# 2 :-> 5
# 3 :-> 6
# 4 :-> 8
r_n = random.randint(low, high_)
range_index = bisect.bisect_left(ranges, r_n)
r = r_n + range_index
for i in xrange(range_index, len(ranges)):
if ranges[i] <= r:
# as many "gaps" we iterate over, as much
# is the true random value (`r`) shifted.
r = r_n + i + 1
elif ranges[i] > r_n:
break
# mark `r` as another "gap" of the original
# [low, high) range.
ranges.insert(i, r)
# Fewer values possible.
high_ -= 1
# `ranges` happens to contain the result.
return ranges[:-1]
I found a quite faster way than having to use the range
function (very slow), and without using random
function from python
(I don´t like the random
built-in library because when you seed it, it repeats the pattern of the random numbers generator)
import numpy as np
nums = set(np.random.randint(low=0, high=100, size=150)) #generate some more for the duplicates
nums = list(nums)[:100]
This is quite fast.
-
"I don´t like the
random
built-in library because when you seed it, it repeats the pattern of the random numbers generator" What? The whole point of providing an explicit seed is to get the same "random" outputs reliably. If you don't seed it (or seed it without passing an argument toseed
) it produces unpredictable values as expected (it seeds from OS-supplied cryptographic randomness). Jun 3, 2022 at 18:50
You can use Numpy library for quick answer as shown below -
Given code snippet lists down 6 unique numbers between the range of 0 to 5. You can adjust the parameters for your comfort.
import numpy as np
import random
a = np.linspace( 0, 5, 6 )
random.shuffle(a)
print(a)
Output
[ 2. 1. 5. 3. 4. 0.]
It doesn't put any constraints as we see in random.sample
as referred here.
-
1
import random
sourcelist=[]
resultlist=[]
for x in range(100):
sourcelist.append(x)
for y in sourcelist:
resultlist.insert(random.randint(0,len(resultlist)),y)
print (resultlist)
-
1Welcome to Stackoverflow. Please explain your answer why and how does it solve the problem so others can understand your answer easily.– octobusApr 29, 2020 at 7:47
-
While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add explanations and give an indication of what limitations and assumptions apply. From Review Apr 29, 2020 at 8:40
Try using...
import random
LENGTH = 100
random_with_possible_duplicates = [random.randrange(-3, 3) for _ in range(LENGTH)]
random_without_duplicates = list(set(random_with_possible_duplicates)) # This removes duplicates
Advatages
Fast, efficient and readable.
Possible Issues
This method can change the length of the list if there are duplicates.
-
This is a very unstable approach, since the user has no control over how final length of the list. Not sure if I can see the use-case for this– ViggoTWJan 4, 2022 at 14:10
If you wish to ensure that the numbers being added are unique, you could use a Set object
if using 2.7 or greater, or import the sets module if not.
As others have mentioned, this means the numbers are not truly random.
If the amount of numbers you want is random, you can do something like this. In this case, length is the highest number you want to choose from.
If it notices the new random number was already chosen, itll subtract 1 from count (since a count was added before it knew whether it was a duplicate or not). If its not in the list, then do what you want with it and add it to the list so it cant get picked again.
import random
def randomizer():
chosen_number=[]
count=0
user_input = int(input("Enter number for how many rows to randomly select: "))
numlist=[]
#length = whatever the highest number you want to choose from
while 1<=user_input<=length:
count=count+1
if count>user_input:
break
else:
chosen_number = random.randint(0, length)
if line_number in numlist:
count=count-1
continue
if chosen_number not in numlist:
numlist.append(chosen_number)
#do what you want here
Edit: ignore my answer here. use python's random.shuffle
or random.sample
, as mentioned in other answers.
import numpy as np
minval, maxval, n_samples = -50, 50, 10
generator = np.random.default_rng(seed=0)
samples = generator.permutation(np.arange(minval, maxval))[:n_samples]
# or, if minval is 0,
samples = generator.permutation(maxval)[:n_samples]
with jax:
import jax
minval, maxval, n_samples = -50, 50, 10
key = jax.random.PRNGKey(seed=0)
samples = jax.random.shuffle(key, jax.numpy.arange(minval, maxval))[:n_samples]
-
Why would you generate a permutaiton of a possibly large number of elements and then only select the first
n_samples
of them? What is your reasoning behind this approach? Can you explain what are the advantages of your approach, compared to any of the large number of existing answers (most of them from 8 years ago)?– kyrillApr 26, 2020 at 15:38 -
actually my answer has similar complexity as other top voted answers and is faster because it uses numpy. other, top-voted methods use
random.shuffle
, which uses Mersenne Twister, qhich is much slower than algos offered by numpy (and probably jax). numpy and jax allow for other random number generation algorithms. jax also allows jit-compiling and differentiation, which can be useful for stochastic differentiation. also, regarding a "possibly large" array, some top voted answers do the exact same thing withrandom.shuffle
, which i don't think is sinful in a relative or even absolute sense May 20, 2020 at 20:32 -
1Not sure what you mean by "
random.shuffle
uses Mersenne Twister" ‒ it's Fisher-Yates shuffle, as mentioned in several answers. It has linear time complexity so it cannot possibly be asymptotically slower than algorithms offered by any other library, numpy or otherwise. If numpy is faster, it is only because it's imlpemented in C, but this does not warrant generating a huge permutation (one which might not even fit into memory), only to choose a few elements from it. There is not a single answer besides yours which does this.– kyrillMay 21, 2020 at 1:32 -
My apologies, I read that python random used Mersenne Twister as it's prng. Do you have a source so I can learn more about Fisher Yates and the role in random.shuffle? May 24, 2020 at 19:57
-
There already are two separate links to Wikipedia on two separate answers here. If Wikipedia is not a good enough source for you, there are 14 references at the end of the article. And then there's Google. Does that help? Oh, and the
random
module is written in Python, so you can easily view its source (tryrandom.__file__
).– kyrillMay 25, 2020 at 8:52
From the CLI in win xp:
python -c "import random; print(sorted(set([random.randint(6,49) for i in range(7)]))[:6])"
In Canada we have the 6/49 Lotto. I just wrap the above code in lotto.bat and run C:\home\lotto.bat
or just C:\home\lotto
.
Because random.randint
often repeats a number, I use set
with range(7)
and then shorten it to a length of 6.
Occasionally if a number repeats more than 2 times the resulting list length will be less than 6.
EDIT: However, random.sample(range(6,49),6)
is the correct way to go.