1

I want to enable the user to search for other users across multiple columns in the user model. Namely the first_name, last_name, and email fields. I'm doing MySQL full text search for now but I'm not convinced that's the right way for me to proceed. Does anybody know how MySQL full text searching stacks up against Solr and other third party providers?

I'm looking at searchify and websolr as potentials. But for 1 table to do full text searching across 3 columns, would it be worth it?

2 Answers 2

3

Searching against the database itself is rather slow. The recommended approach is using a search engine, like Solr, Whoosh, etc to generate the indexes. Haystack is a very useful django app that let's you abstract the search engines and have templates to use when indexing.

So with the template you can have a template like:

{{user.first_name}}
{{user. last_name}}
{{user.email}}

And it will generate the search results you're looking for.

3
  • Question though...if I want to use websolr or any of the alternatives, can it automatically index tables from MySQL database automatically? Cannot find any information on it...
    – KVISH
    Jul 24, 2012 at 13:59
  • Yeah that's the good thing about it. Take a look at the Haystack documentation. Basically you would create a template like the one I posted, and then run an indexing command; after that, you're ready to go.
    – leonsas
    Jul 24, 2012 at 15:00
  • Also, for development purposes, I would recommend the Whoosh search engine, which is completely written in Python, and I feel it's much easier to setup and 'play' with it.
    – leonsas
    Jul 24, 2012 at 15:08
2

This is how I do it

#search.py
import re

from django.db.models import Q

def normalize_query(query_string,
                    findterms=re.compile(r'"([^"]+)"|(\S+)').findall,
                    normspace=re.compile(r'\s{2,}').sub):
    return [normspace(' ', (t[0] or t[1]).strip()) for t in findterms(query_string)] 

def get_query(query_string, search_fields):
    query = None
    terms = normalize_query(query_string)
    for term in terms:
        or_query = None
        for field_name in search_fields:
            q = Q(**{"%s__icontains" % field_name: term})
            if or_query is None:
                or_query = q
            else:
                or_query = or_query | q
        if query is None:
            query = or_query
        else:
            query = query & or_query
    return query

And the view:

# views.py
from django.shortcuts import render_to_response
from django.contrib.auth.models import User

def search(request):
    query = request.GET.get('q', '')
    if query:
        entry_query = get_query(query, ['first_name', 'last_name', 'email'])
        users = User.objects.filter(entry_query).order_by('-pub_date')
    else:
        entries_list = []
    return render_response(request, 'blog/list.html', {'entries': entries_list})
1
  • I don't have any high traffic site to tell you, but in a standar traffic site works actually great!
    – nicowernli
    Jul 24, 2012 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.