2

My search engine uses the following function to calculate relevancy.

private static int calculateScore(String result, String searchStr, int modifier) 
{
    String[] resultWords = result.split(" ");
    String[] searchWords = searchStr.split(" ");

    int score = 0;
    for (String searchWord : searchWords) 
    {
        for (String resultWord : resultWords) 
        {
            if (resultWord.equals(searchWord))
                score += 10;
            else if (resultWord.startsWith(searchWord))
                score += 4;
            else if (resultWord.endsWith(searchWord))
                score += 3;
            else if (resultWord.contains(searchWord))
                score += 1;
        }

    }
    return score;
}

Nothing fancy, and I haven't been given enough hours to do anything fancy either, but are there any simple improvements I can do to make the function better at upping the relevant stuff, and keeping the irrelevant stuff down? No need to remark on speed optimizations, this is just the "functional part" of the function :)

Thanks.

1

7 Answers 7

9

Not sure if it counts as fancy, but a soundex comparison, presumably earning a +1 score on your scale, will grant a little relevance to typographical near misses and homophones.

I'd suggest using a stop word list to either prevent or radically reduce relevancy granted from common words. If someone is searching for "the horse is on the roof", you want to grant relevancy to "horse" and "roof", not massively upscore everything containing "the".

Another easy boost is to grant a whole bunch of relevancy to a result that contains the entire search string in order (case and punctuation insensitive).

5

The classic vector space model is actually standard nice and rather simple (in implementation) solution.

http://www.miislita.com/term-vector/term-vector-3.html

2
  • 2
    Wow, that made my head spin. I wish I was more mathematically inclined :)
    – Ace
    May 27, 2009 at 16:04
  • 1
    Actually, this looks pretty cool. I'll make this the accepted answer if someone can explain it to a math-imbecile like me. =)
    – Ace
    May 29, 2009 at 11:10
2

You should also probably normalize case before you do your equals.

2

You could add +1 to the score if the Levenshtein Distance is below some threshold.

2

If you want to make your search algorithm a little bit more sophisticated, look at cosine similarity. It's pretty easy to implement and works pretty well in practice.

0
1

You could:

  • Cull out noise words (the, a, at, etc...)
  • Change weight for searchable text relevancy - if a search word occurs in a document's title and its body, the title is probably more relevant.
  • Depending on your resultWord text, consider adding weight for words earlier or later in the text.
  • Make unique search word matches worth more (you'd have to determine what makes a word unique).
0

Obvious, but you'd want to lowercase everything.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.