1

I have an application (Node/Angular) that I'm creating where I'm trying to rank users based on overall performance across two metrics. There are two metrics used to track the users we are using are the following:

  1. Units Produced (ranges between 0 - 6000)
  2. Rate of production = [ Units Produced ] / [ Labor Hours ] (ranges between 0 - 100)

However, ranking users explicitly by either of these variables doesn't make sense, because it creates some strange incentives/behaviors.

For instance, it is possible to have a really high Rate of Production, but a super low number of total number of units produced by working really hard over a short period of time. Alternatively, you can have a very high number of Units Produced, but it may be due to the fact that they worked overtime, and thus were able to produce more units than anyone else just due to the fact that they had longer to work, and they could have a low Rate of Production.

Does anyone have experience designing these types of scoring systems? How have you handled it?

4
  • This is not an angular or node specific question. Please use other tags in future for these types of questions.
    – ggilberth
    Aug 20, 2015 at 15:53
  • @ggilberth I'd love to - any suggestions? I couldn't find a tag that seemed to fit...
    – Jonathan
    Aug 20, 2015 at 16:01
  • Algorithm I think would be a good one.
    – ggilberth
    Aug 20, 2015 at 16:02
  • 1
    Done! Thanks for the suggestion
    – Jonathan
    Aug 20, 2015 at 16:03

3 Answers 3

2

First, I would recommend to bring them on the same scale. E.g. divide Units produced by 60.

Then, if you are fine with equal weights, there are three common simple choices:

  1. Add the scores

  2. Multiply the scores (equal to adding logs of each)

  3. Take the minimum of the two scores

Which of the ones is best, depends on to what extent you want it to be a measure of combined good results. In your case, I would recommend you to multiply and put a scale on the resulting product.

If you want to go a little more complex and weigh or play around with how much to reward separate vs joint scores, you can use the following formula:

V = alpha * log_b[Units Produced / 60] + (1-alpha) * log_b[Rate of Production],

where alpha determines the weighting of one vs the other and the base of the logarithmic function determines to what extent a joint success is rewarded.

3
  • I like your idea of adding the logs of each! I had been multiplying the two together as a trial, but getting some results that still didn't feel quite right since one of the factors still greatly influenced the overall result. But, given your explanation here, it looks like I need to normalize the two numbers to allow them to be weighted equally. Thanks for your help - I'll give that a shot!
    – Jonathan
    Aug 20, 2015 at 16:50
  • Multiplying is not a good idea because one is already a multiple of the other. Aug 20, 2015 at 16:52
  • Why would that matter? Aug 20, 2015 at 17:05
0

I did something very similar I found it valuable to break them into leagues or tiers, for example using Units Produced as a base.

  • Novice = 100 Units Produced
  • Beginner = 500 Units Produced
  • Advanced = 2000 Units Produced
  • Expert = 4000 Units Produced

Putting this into a useable object

var levels = [
    {id: 1, name: "Novice", minUnits: 100, maxUnits: 599 },
    {id: 2, name: "Beginner", minUnits: 500, maxUnits: 1999 },
    {id: 3, name: "Advanced", minUnits: 2000, maxUnits: 3999 },
    {id: 4, name: "Expert", minUnits: 4000, maxUnits: 6000 }
]

You can then use your Rate of production to multiply by a weighted value inside the levels, you can determine what this is. You can play with the values to make it as hard or as easy as you want.

1
  • Interesting idea - however I think this allows for loopholes to gain rank by compensating for inefficient work patterns with quantity. Only ranking tiers by total number of units allows people to make up for their inefficient work patterns by just working longer. And if I'm paying people by the hour, I'd prefer that they don't work longer - I want them to be efficient as well.
    – Jonathan
    Aug 20, 2015 at 16:03
0

You can do a combination with

SCORE = 200/( K_1/x_1 + K_2/x_2 )

// x_1 : Score 1
// x_2 : Score 2
// K_1 : Maximum of Score 1
// K_2 : Maximum of Score 2

Of course be carefull when dividing by zero. If either x_1 or x_2 are zero then SCORE=0. If x_1=K_1 and x_2=K_2 then SCORE=100 (maximum)

Otherwise the score is somewhere in between. If x_1/K_1 = x_2/K_2 = z then SCORE = 100*z

This weighs the lower score more such that you get rewarded when raising one of the two scores (unlike a minimum of the two scenarios) but not as much as raising both.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.