0

I have the following dataframe

 bootstrap  cluster_main    cluster_b   distance
    1   0   Cluster 0   Cluster 1   0.002016
    15  0   Cluster 0   Cluster 3   0.001282
    4   0   Cluster 1   Cluster 0   0.000772
    10  0   Cluster 2   Cluster 2   0.000990
    26  1   Cluster 0   Cluster 2   0.001034
    16  1   Cluster 2   Cluster 0   0.000159
    31  1   Cluster 3   Cluster 3   0.000889
    21  1   Cluster 1   Cluster 1   0.000961
    35  2   Cluster 0   Cluster 3   0.099427
    36  2   Cluster 1   Cluster 0   0.067036
    43  2   Cluster 2   Cluster 3   0.102834
    45  2   Cluster 3   Cluster 1   0.069814

I would like to find the bootstraps for which there is no one-to-one matching between cluster_main and cluster_b.

In the above example, the output should be 2 and 0, because Cluster 3 in cluster_b column for bootstrap 2, is "matched" twice and the same happens for Cluster 0 in the cluster_main column for bootstrap 0

5
  • Outut is not 2m because in 10 0 Cluster 2 Cluster 2 0.000990 ? There is match Cluster 2
    – jezrael
    Sep 18, 2020 at 8:02
  • 1
    the output is the bootstrap number, not the cluster number. To put it in other words, for every bootstrap, that there is not a one-to-one match of cluster_main to cluster_b, i would like the bootstrap number. By one to one, I mean a that no Cluster from cluster_main is matched twice to a Cluster from cluster_b, and vice versa
    – quant
    Sep 18, 2020 at 8:06
  • Is possible also say need test if all values cluster_b are matched with cluster_b per bootstrap and it not, get this values? Answer was edited.
    – jezrael
    Sep 18, 2020 at 8:08
  • I think the logic can be: for every bootstrap, cluster_main AND cluster_b should have 4 unique values, otherwise show the bootstrap number
    – quant
    Sep 18, 2020 at 8:15
  • 1
    Answer was edited for compare sorted values of column or sets
    – jezrael
    Sep 18, 2020 at 8:20

1 Answer 1

1

I believe you need:

#compared sorted values
#f = lambda x: sorted(x['cluster_main']) == sorted(x['cluster_b'])
#comppred sets
#f = lambda x: set(x['cluster_main']) == set(x['cluster_b'])
m = df.groupby('bootstrap').apply(f)
print (m)
bootstrap
0    False
1     True
2    False
dtype: bool
bootstrap
0    False
1     True
2    False
dtype: bool

out = m.index[~m]
print (out)
Int64Index([0, 2], dtype='int64', name='bootstrap')

EDIT: I realised first solution was same like comopared sets, so removed.

Here is possible see difference:

print (df)
    bootstrap cluster_main  cluster_b  distance
1           0    Cluster 0  Cluster 1  0.002016
15          0    Cluster 0  Cluster 1  0.001282
4           0    Cluster 1  Cluster 0  0.000772
10          0    Cluster 2  Cluster 2  0.000990
26          1    Cluster 2  Cluster 0  0.001034
16          1    Cluster 0  Cluster 2  0.000159
31          1    Cluster 3  Cluster 3  0.000889
21          1    Cluster 1  Cluster 1  0.000961
35          2    Cluster 0  Cluster 0  0.099427
36          2    Cluster 2  Cluster 2  0.067036
43          2    Cluster 2  Cluster 3  0.102834
45          2    Cluster 3  Cluster 2  0.069814


#compared sorted values
f = lambda x: sorted(x['cluster_main']) == sorted(x['cluster_b'])
m = df.groupby('bootstrap').apply(f)
print (m)
bootstrap
0    False
1     True
2     True
dtype: bool

f = lambda x: set(x['cluster_main']) == set(x['cluster_b'])
m = df.groupby('bootstrap').apply(f)
print (m)
bootstrap
0    True
1    True
2    True
dtype: bool
5
  • I do not think this would work. I edited a bit the input df, but the output should be again [0,2], where as the above code will give [0,1,2]
    – quant
    Sep 18, 2020 at 8:13
  • I do not understand the difference between the 2 approaches, if I am honest
    – quant
    Sep 18, 2020 at 8:21
  • @quant - I change data for explain it
    – jezrael
    Sep 18, 2020 at 8:22
  • 1
    @quant - added sampled ata, can you check it?
    – jezrael
    Sep 18, 2020 at 8:29
  • 1
    I did check it, in my full dataset as well. It works like a charm, and the logic is clear to me, indeed sorted is the way to go
    – quant
    Sep 18, 2020 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.