I have a df like so (actual df has 4.5 mil rows, 23 cols):
group feature col1 col2 col3
g1 f1 1 10 100
g1 f1 11 9 1000
g1 f2 0 8 200
g2 f1 2 7 330
g2 f2 3 7 331
g2 f3 1 7 100
g3 f1 1 6 101
g3 f1 5 9 100
g3 f1 1 8 100
I want to add two new "rank" cols for each col in my df. I will evaluate different cols differently, such as sum, mean, max, etc. For ease of explanation I've broken the problem out into two separate problems below.
I have been advised here to use .loc
and not use groupby
, but any solution that works is fine. I've tried both and had little success (see here)
The first rank col will rank each feature on the values in col1, col2, and col3 within each group.
At an intermediate stage it would look something like this:
group feature col1 col1_sum col1_rank col2 col2_avg col2_rank col3 col3_max col3_rank
g1 f1 1 12 1 10 9.5 1 100 1000 1
g1 f1 11 9 1000
g1 f2 0 0 2 8 8 2 200 200 2
g2 f1 2 2 2 7 7 1 330 330 2
g2 f2 3 3 1 7 7 1 331 331 1
g2 f3 1 1 3 7 7 1 100 100 3
g3 f1 1 7 1 6 7.67 1 101 101 1
g3 f1 5 9 100
g3 f1 1 8 100
It will output this:
group feature col1_rank col2_rank col3_rank
g1 f1 1 1 1
g1 f2 2 2 2
g2 f1 2 1 2
g2 f2 1 1 1
g2 f3 3 1 3
g3 f1 1 1 1
The second rank col will rank each group by feature on the values in col1, col2, and col3 against all other groups.
At an intermediate stage it would look something like this:
group feature col1 col1_sum col1_rank col2 col2_avg col2_rank col3 col3_max col3_rank
g1 f1 1 12 1 10 9.5 1 100 1000 1
g1 f1 11 9 1000
g2 f1 2 2 3 7 7 3 330 330 2
g3 f1 1 7 2 6 7.67 2 101 101 3
g3 f1 5 9 100
g3 f1 1 8 100
g1 f2 0 0 2 8 8 1 200 200 2
g2 f2 3 3 1 7 7 2 331 331 1
g2 f3 1 1 1 7 7 1 100 100 1
It will output this:
group feature col1_rank col2_rank col3_rank
g1 f1 1 1 1
g2 f1 3 3 2
g3 f1 2 2 3
g1 f2 2 1 2
g2 f2 1 2 1
g2 f3 1 1 1