3

I used this SO question to enable full text search on a mysql db in Django application.

# models.py

class CaseSnapshot(BaseModel):
    text = models.TextField(blank=True)


class Search(models.Lookup):
    lookup_name = "search"

    def as_mysql(self, compiler, connection):
        lhs, lhs_params = self.process_lhs(compiler, connection)
        rhs, rhs_params = self.process_rhs(compiler, connection)
        params = lhs_params + rhs_params
        return f"MATCH (%s) AGAINST (%s IN BOOLEAN MODE)" % (lhs, rhs), params


models.TextField.register_lookup(Search)

Because Django does not support full text search on mysql, we have to add our index, so we modify the generated migration file:

# Generated by Django 2.2 on 2020-04-28 03:41

from django.db import migrations, models

# Table details
table_name = "by_api_casesnapshot"
field_name = "text"
index_name = f"{table_name}_{field_name}_index"


class Migration(migrations.Migration):

    dependencies = [("by_api", "0033_add_tag_color")]

    operations = [
        migrations.CreateModel(...), # As auto-generated
        migrations.RunSQL(
            f"CREATE FULLTEXT INDEX {index_name} ON {table_name} ({field_name})",
            f"DROP INDEX {index_name} ON {table_name}",
        ),
    ]

Checking the Work

# dbshell

mysql> SHOW INDEX FROM by_api_casesnapshot;

+---------------------+--------------------------------+-------------+-----+------------+-----+
| Table               | Key_name                       | Column_name | ... | Index_type | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
| by_api_casesnapshot | PRIMARY                        | id          | ... | BTREE      | ... |
| by_api_casesnapshot | by_api_casesnapshot_text_index | text        | ... | FULLTEXT   | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
#shell 
>>> snap = CaseSnapshot.objects.create(text="XXX")

# Regular query first
>>> CaseSnapshot.objects.filter(text__contains="X") 
<QuerySet [<CaseSnapshot pk=1>]>

# Now test custom search lookup
>>> CaseSnapshot.objects.filter(text__search="XXX")
<QuerySet [<CaseSnapshot pk=1>]>

>>> CaseSnapshot.objects.filter(text__search="X*")
<QuerySet [<CaseSnapshot pk=1>]>

>>> CaseSnapshot.objects.filter(text__search="X*").query.__str__()

'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (X* IN BOOLEAN M
ODE)'

All that works exactly as intended so far.

Issue

Here is where things fail - all queries using the __search lookups during test runs return an queryset empty set, even thought the generated query is identical.

Tests

from django.test import TestCase
from by_api.models import CaseSnapshot


class TestModels(TestCase):
    def test_search(self):
        CaseSnapshot.objects.create(text="XXX")
        rv1 = CaseSnapshot.objects.filter(text__contains="XXX")
        # Passes
        self.assertEqual(rv1.count(), 1)

        rv2 = CaseSnapshot.objects.filter(text__search="XXX")
        # Fails - count i
        self.assertEqual(rv2.count(), 1)


I tried to debug this with pdb inside the test run, and note the generated query is identical to the one produced above in the shell environment:

>>> rv2.query.__str__())
'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (XXX IN BOOLEAN MODE)'

Question

Why does use of __search lookup during tests produce empty queryset?

What I have tried

Update

I just noticed if I use unittest.TestCase instead of django.test.TestCase the test passes - could this be related to how django's test case wraps db transactions?

1 Answer 1

6

Answer from Adam Chainz from django-mysql library:

InnoDB doesn't make changes to full text indexes until commits. You can use TransactionTestCase to get around this.

I change the test class to use TransactionTestCase and the tests now pass.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.