8

Lets say I have a dataframe like below. What I want is that, if a number between columns a,b,c appear most then it should output that number or if the all three numbers are different then take the number of a. For example, in first row, 1 appears most among 1 and 5 then output in d is 1. But in second row, all three numbers 11, 2, 7 of column a,b,c are different, output is the value of column a(11), so output in d is 11

list   a  b   c  
 1     1  5   1 
11    11  2   7 
 0     0  0   0 
 9     5  9   5 
 8     8  2   7  

Expected output

list   a  b   c  d 
 1     1  5   1  1
11    11  2   7  11
 0     0  0   0  0
 9     5  9   5  5
 8     8  2   7  8 
7
  • 3
    I’ve never tried it but df.mode(1).iloc[:, 0]
    – piRSquared
    Commented Jan 23, 2018 at 1:21
  • 1
    @piRSquared Wow... had no idea we had a mode function too. (also, it works!)
    – cs95
    Commented Jan 23, 2018 at 1:23
  • @cᴏʟᴅsᴘᴇᴇᴅ thanks for checking (-:
    – piRSquared
    Commented Jan 23, 2018 at 1:28
  • 1
    the iloc doesn't quite work as OP requests though. mode seems to return sort ordered rather than original order when there is not a strict mode. e.g. for second row, mode returns [2,7,11] rather than [11,2,7]
    – JohnE
    Commented Jan 23, 2018 at 1:35
  • @JohnE that stinks.
    – piRSquared
    Commented Jan 23, 2018 at 1:36

4 Answers 4

1

scipy calculates mode, but I am surprised not to find this in numpy.

import pandas as pd
import numpy as np
from scipy import stats

df = pd.DataFrame([[1, 1, 5, 1],
                   [11, 11, 2, 7],
                   [0, 0, 0, 0],
                   [9, 5, 9, 5],
                   [8, 8, 2, 7]],
                  columns=['list', 'a', 'b', 'c'])

df['d'], df['count'] = stats.mode(df[['a', 'b', 'c']].values, axis=1)
df.loc[df['count'] == 1, 'd'] = df['a']
df = df.drop('count', 1)
1

You can using value_counts

df.iloc[:,1:].apply(lambda x : x.value_counts().index[0] if x.value_counts().iloc[0]>1 else x['a'] ,1)
Out[1046]: 
0     1
1    11
2     0
3     5
4     8
dtype: int64
1

Here is my bincount solution

Data
Note that this is different from OP's to point out that it works as expected.

   list   a  b  c
0     1   5  1  1
1    11  11  2  7
2     0   0  0  0
3     9   5  9  5
4     8   8  2  7

Solution

v = df.values[:, 1:]

f, u = pd.factorize([(i, e) for i, row in enumerate(v) for e in row])

counts = np.bincount(f)[f].reshape(v.shape)

x = (counts == counts.max(1, keepdims=1)).argmax(1)
y = np.arange(v.shape[0])

df.assign(d=v[y, x])

   list   a  b  c   d
0     1   5  1  1   1
1    11  11  2  7  11
2     0   0  0  0   0
3     9   5  9  5   5
4     8   8  2  7   8

Details

Get numpy array of just the values we want.

v = df.values[:, 1:]

Use enumerate and comprehension to create a list of tuples. Each rows values will be distinct from other rows because I'm placing an identifier in the first position of the tuple for each row. Namely the value from enumerate. I then pass these into Pandas' factorize function in order to place into Numpy's bincount.

f, u = pd.factorize([(i, e) for i, row in enumerate(v) for e in row])

Now I use bincount on f and slice it with f to get an array of the same size, but now filled with count values.

counts = np.bincount(f)[f].reshape(v.shape)

I locate the maximum values and slice the original array to get what those values where.

x = (counts == counts.max(1, keepdims=1)).argmax(1)
y = np.arange(v.shape[0])

Note that if all values are the same or if there are multiple modes, argmax will pick the first one. When all are the same, this is column a.

df.assign(d=v[y, x])
0

As @piRSquared suggested, we can use mode function in pandas.

df["d"] = np.where(df.apply(lambda x: x.nunique() == 3, 1), 
                   df["a"], 
                   df.mode(1).loc[:,0])

Another method (for fun)

df["d"] = np.where(df['b'] == df['c'], df['b'], df['a']) 

Pseudocode explanation

if b == c:
    choose b because b and c are both the mode
else:  # when b != c
   if a == c or a == b:
        choose a because a is the mode
   else: # a != b != c
        choose a because all are different 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.