3

My goal is to understand Average Precision at K, and Recall at K. I have two lists, one is predicted and other is actual (ground truth)

lets call these two lists as predicted and actual. Now I want to do precision@k and recall@k.

Using python I implemented Avg precision at K as follows:

def apk(actual, predicted, k=10):
    """
    Computes the average precision at k.

    This function computes the average precision at k between two lists of items.

    Parameters
    ----------
    actual: list
            A list of elements that are to be predicted (order doesn't matter)
    predicted : list
            A list of predicted elements (order does matter)
    k: int, optional

    Returns
    -------
    score : double
            The average precision at k over the input lists

    """
    if len(predicted) > k:
        predicted = predicted[:k]

    score = 0.0
    num_hits = 0.0

    for i,p in enumerate(predicted):
        if p in actual and p not in predicted[:i]:
            num_hits += 1.0
            score += num_hits / (i + 1.0)

    if not actual:
        return 1.0
    if min(len(actual), k) == 0:
        return 0.0
    else:
        return score / min(len(actual), k)

lets assume that our predicted has 5 strings in following order: predicted = ['b','c','a','e','d'] andactual = ['a','b','e']since we are doing @k would the precision@k is same asrecall@k? If not how would I dorecall@k`

If I want to do f-measure (f-score) what would be the best route to do for above mention list?

1 Answer 1

8

I guess, you've already checked wiki. Based on its formula, the 3rd and the biggest one (after the words 'This finite sum is equivalent to:'), let's see at your example for each iteration:

  1. i=1 p = 1
  2. i=2 rel = 0
  3. i=3 p = 2/3
  4. i=4 p = 3/4
  5. i=5 rel = 0

So, avp@4 = avp@5 = (1 + 0.66 + 0.75) / 3 = 0.805; avp@3 = (1 + 0.66) / 3 and so on.

Recall@5 = Recall@4 = 3/3 = 1; Recall@3 = 2/3; Recall@2 =Recall@1 = 1/3

Below is the code for precision@k and recall@k. I kept your notation, while it seems to be more common to use actual for observed/returned value and expected for ground truth (see for example JUnit defaults).

def precision(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(k)
    return result

def recall(actual, predicted, k):
    act_set = set(actual)
    pred_set = set(predicted[:k])
    result = len(act_set & pred_set) / float(len(act_set))
    return result
1
  • Hi! Are you sure this is the correct implementation? Here is the example: y_true = np.array([1,1,1,1]) preds = np.array([1,1,1,1]) precision(y_true, preds, k=3) output: 0.3333 Commented Nov 2, 2020 at 6:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.