30

I have a 4D array training images, whose dimensions correspond to (num_images, channels, width, height). I also have a 2D target labels whose dimensions correspond to (num_images, class_number). When training, I want to randomly shuffle the data by using random.shuffle, but how can I keep the labels shuffled in the same order as my images?

6 Answers 6

92
from sklearn.utils import shuffle
import numpy as np

X = np.array([
  [0, 0, 0],
  [1, 1, 1], 
  [2, 2, 2], 
  [3, 3, 3], 
  [4, 4, 4]
])
y = np.array([0, 1, 2, 3, 4])
X, y = shuffle(X, y)
print(X)
print(y)

Output:

[[1 1 1]
 [3 3 3]
 [0 0 0]
 [2 2 2]
 [4 4 4]] 

[1 3 0 2 4]
2
  • 8
    To help readers understand your solution, consider including a short description of your code and how it solves the posted question Commented Oct 8, 2018 at 1:11
  • 31
    @HenryWoody I think the the code is self-explanatory.
    – Foreever
    Commented Nov 2, 2018 at 2:40
14

There is another easy way to do that. Let us suppose that there are total N images. Then we can do the following:

from random import shuffle

ind_list = [i for i in range(N)]
shuffle(ind_list)
train_new  = train[ind_list, :,:,:]
target_new = target[ind_list,]
2
  • 2
    Instead of [i for i in range(N)] you could use list(range(N)).
    – krenerd
    Commented Feb 26, 2021 at 1:42
  • This is a good solution for shuffle more than 2 data structures. Thanks Commented Jun 23, 2022 at 16:36
10

If you want a numpy-only solution, you can just reindex the second array on the first, assuming you've got the same image numbers in both:

In [67]: train = np.arange(20).reshape(4,5).T

In [68]: target = np.hstack([np.arange(5).reshape(5,1), np.arange(100, 105).reshape(5,1)])

In [69]: train
Out[69]:
array([[ 0,  5, 10, 15],
       [ 1,  6, 11, 16],
       [ 2,  7, 12, 17],
       [ 3,  8, 13, 18],
       [ 4,  9, 14, 19]])

In [70]: target
Out[70]:
array([[  0, 100],
       [  1, 101],
       [  2, 102],
       [  3, 103],
       [  4, 104]])

In [71]: np.random.shuffle(train)

In [72]: target[train[:,0]]
Out[72]:
array([[  2, 102],
       [  3, 103],
       [  1, 101],
       [  4, 104],
       [  0, 100]])

In [73]: train
Out[73]:
array([[ 2,  7, 12, 17],
       [ 3,  8, 13, 18],
       [ 1,  6, 11, 16],
       [ 4,  9, 14, 19],
       [ 0,  5, 10, 15]])
1

If you're looking for a sync/ unison shuffle you can use the following func.

def unisonShuffleDataset(a, b):
    assert len(a) == len(b)
    p = np.random.permutation(len(a))
    return a[p], b[p]

the one above is only for 2 numpy. One can extend to more than 2 by adding the number of input vars on the func. and also on the return of the function.

0

Depending on what you want to do, you could also randomly generate a number for each dimension of your array with

random.randint(a, b)  #a and b are the extremes of your array

which would select randomly amongst your objects.

1
  • As random.randint(a, b) does not guarantee that the number generated is not the same as previously generated before, so there would a manual work to keep track that every time you are generating a unique number, till all the data is covered.
    – warriorUSP
    Commented Mar 15, 2020 at 19:36
0

Use the same seed to build the random generator multiple times to shuffle different arrays:

>>> seed = np.random.SeedSequence()
>>> arrays = [np.arange(10).repeat(i).reshape(10, -1) for i in range(1, 4)]
>>> for ar in arrays:
...     np.random.default_rng(seed).shuffle(ar)
...
>>> arrays
[array([[1],
        [2],
        [7],
        [8],
        [0],
        [4],
        [3],
        [6],
        [9],
        [5]]),
 array([[1, 1],
        [2, 2],
        [7, 7],
        [8, 8],
        [0, 0],
        [4, 4],
        [3, 3],
        [6, 6],
        [9, 9],
        [5, 5]]),
 array([[1, 1, 1],
        [2, 2, 2],
        [7, 7, 7],
        [8, 8, 8],
        [0, 0, 0],
        [4, 4, 4],
        [3, 3, 3],
        [6, 6, 6],
        [9, 9, 9],
        [5, 5, 5]])]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.