197

I want to remove null=True from a TextField:

-    footer=models.TextField(null=True, blank=True)
+    footer=models.TextField(blank=True, default='')

I created a schema migration:

manage.py schemamigration fooapp --auto

Since some footer columns contain NULL I get this error if I run the migration:

django.db.utils.IntegrityError: column "footer" contains null values

I added this to the schema migration:

    for sender in orm['fooapp.EmailSender'].objects.filter(footer=None):
        sender.footer=''
        sender.save()

Now I get:

django.db.utils.DatabaseError: cannot ALTER TABLE "fooapp_emailsender" because it has pending trigger events

What is wrong?

2
  • 2
    This question is similar: stackoverflow.com/questions/28429933/… and had answers that were more useful to me. Apr 25, 2017 at 16:35
  • 1
    I got the same problem with Postgres v10 (but not with Postgres v.12). Problem solved, by adding a separate migration file.
    – nik_m
    Aug 11, 2021 at 9:04

10 Answers 10

210

Another reason for this maybe because you try to set a column to NOT NULL when it actually already has NULL values.

4
  • 12
    To address this you can either use a data migration or manually (manage.py shell) go in and update non-compliant values
    – mgojohn
    Oct 26, 2014 at 23:13
  • @mgojohn How do you do that? Jul 28, 2015 at 23:42
  • 1
    @pyramidface If you aren't too picky, you can just update the null values at the django shell. If you're looking for something more formal and testable, it depends on what versions you're using. If you use south, see: south.readthedocs.org/en/latest/tutorial/part3.html and if you use django's migrations, see the "data migrations" section here: docs.djangoproject.com/en/1.8/topics/migrations
    – mgojohn
    Jul 29, 2015 at 0:23
  • 2
    I'm a huge fan of Django but obviously error messages like this - that are COMPLETE red herrings - just goes to show the complexity of such platforms! Feb 15, 2021 at 0:39
200

Every migration is inside a transaction. In PostgreSQL you must not update the table and then alter the table schema in one transaction.

You need to split the data migration and the schema migration. First create the data migration with this code:

 for sender in orm['fooapp.EmailSender'].objects.filter(footer=None):
    sender.footer=''
    sender.save()

Then create the schema migration:

manage.py schemamigration fooapp --auto

Now you have two transactions and the migration in two steps should work.

10
  • 15
    PostgreSQL probably changed its behaviour regarding such transactions, as I managed to run a migration with both data and schema changes on my dev machine (PostgreSQL 9.4) while it failed on the server (PostgreSQL 9.1). Jan 24, 2015 at 17:49
  • 2
    If using a RunPython operation in the migration for the data migration, you just need to make sure it's the last operation. Django knows that if the RunPython operation is last, to open its own transaction.
    – Dougyfresh
    Sep 10, 2019 at 19:42
  • 2
    @Dougyfresh is this a documented feature of django?
    – guettli
    Sep 11, 2019 at 8:01
  • 7
    Django's migration also support Migration.atomic = False which allows you to not have to spit the file (as mentioned in the answer).
    – jnns
    Aug 28, 2020 at 7:21
  • 2
    @jnns thanks, setting atomic to False solved my problem
    – DataGreed
    Nov 6, 2020 at 0:59
38

At the operations I put SET CONSTRAINTS:

operations = [
    migrations.RunSQL('SET CONSTRAINTS ALL IMMEDIATE;'),
    migrations.RunPython(migration_func),
    migrations.RunSQL('SET CONSTRAINTS ALL DEFERRED;'),
]
3
  • 3
    Better to use SeparateDatabaseAndState
    – bdoubleu
    Jul 6, 2020 at 17:24
  • @bdoubleu +++ thank you! The original answer is a kind of dirty hack. May 22, 2023 at 16:17
  • Thanks! Found my way here for a different reason, but this solved the same named issue.
    – Andrew
    May 25, 2023 at 3:09
28

If you are adding a non-nullable field, you need to do it in two migrations:

  1. AddField and RunPython to populate it
  2. AlterField to change the field to be non-nullable

Explanation

On PostgreSQL and SQLite, this problem can occur if you have a sufficiently complex RunPython command combined with schema alterations in the same migration. For example, if you are adding a non-nullable field, the typical migration steps for this is:

  1. AddField to add the field as nullable
  2. RunRython to populate it
  3. AlterField to change the field to be non-nullable

On SQLite and Postgres, this can cause problems because the whole thing is being done in one transaction.
The Django docs have a specific warning about this:

On databases that do support DDL transactions (SQLite and PostgreSQL), RunPython operations do not have any transactions automatically added besides the transactions created for each migration. Thus, on PostgreSQL, for example, you should avoid combining schema changes and RunPython operations in the same migration or you may hit errors like OperationalError: cannot ALTER TABLE "mytable" because it has pending trigger events.

If this is the case, the solution is to separate your migration into multiple migrations. In general, the way to split is to have a first migration containing the steps up through the run_python command and the second migration containing all the ones after it. Thus, in the case described above, the pattern would be the AddField and RunPython in one migration, and the AlterField in a second.

1
  • Thank you for pasting the link and quote from the Django docs!
    – Krzysiek
    Aug 30, 2022 at 21:01
11

Have just hit this problem. You can also use db.start_transaction() and db.commit_transaction() in the schema migration to separate data changes from schema changes. Probably not so clean as to have a separate data migration but in my case I would need schema, data, and then another schema migration so I decided to do it all at once.

2
  • 10
    The problem with this solution is this: What happens if your migration fails after db.commit_transaction()? I prefere to use three migrations, if you need this: schema-mig, data-mig, schema-mig.
    – guettli
    Apr 22, 2013 at 13:42
  • 8
    See: django.readthedocs.io/en/latest/ref/migration-operations.html On databases that do support DDL transactions (SQLite and PostgreSQL), RunPython operations do not have any transactions automatically added besides the transactions created for each migration. Thus, on PostgreSQL, for example, you should avoid combining schema changes and RunPython operations in the same migration or you may hit errors like OperationalError: cannot ALTER TABLE "mytable" because it has pending trigger events. Feb 1, 2018 at 10:20
6

@Zags answer is correct, this error usually comes when you mix schema alteration and data manipulation in the same migration, which correspond to a unique transaction by default.

Another solution to solve that issue is to manually set atomic = False to the Migration class. As a result, no transaction is created to run operations, each one is run sequentially and immediatly applied to the database.

class Migration(migrations.Migration):
    atomic = False

    dependencies = [
        ('application', '00XX_previous_migration'),
    ]

    operations = [
        migrations.AddField(
            model_name='foo',
            name='bar',
            field=models.CharField(max_length=255),
        ),
        RunPython(migrate_data, reverse_code=reverse_migrate_data, atomic=True),
        migrations.RemoveField(
            model_name='foo',
            name='baz',
        ),
    ]

Tip: If the RunPython operation perform multiple save(), you can call it with atomic=True to create a transaction for the whole operation.

2
3

In my case I've got

  1. AddField
  2. RunPython
  3. RemoveField

Then I just moved the last RemoveFied to the new migration file, that fixed the problem

2

You are altering the column schema. That footer column can no longer contain a blank value. There are most likely blank values already stored in the DB for that column. Django is going to update those blank rows in your DB from blank to the now default value with the migrate command. Django tries to update the rows where footer column has a blank value and change the schema at the same time it seems (I'm not sure).

The problem is you can't alter the same column schema you are trying to update the values for at the same time.

One solution would be to delete the migrations file updating the schema. Then, run a script to update all those values to your default value. Then re-run the migration to update the schema. This way, the update is already done. Django migration is only altering the schema.

1
  • 2
    Running some script is not really an option for me. I have several instances of the database and the continuous deployment process just calls "manage.py migrate". This question is already valid answers which work fine.
    – guettli
    Apr 28, 2020 at 14:19
0

step 1)the solution is to remove the latest migration from the migration folder and remove the latest added fields in models.

step 2)then again makemigration and migrate

step 3)At the last add the field again that has been removed in the first step

step 4)then again makemigration and migrate

Problem solved

0

The likely cause of this is rooted in your model having check constraints on it via the Meta.constraints option. Check constraints (in Postgres) are implemented as triggers that execute at the end of a transaction. By default, a Django migration file encloses all migrations operations from a file in a single transaction. This means that other ALTER TABLE operations may happen before your triggers fire, a situation that Postgres can't handle.

In order to get around this issue, you can do like another answer suggested:

operations = [
    migrations.RemoveField(...). # Do your normal table operations
    migrations.RunPython(migration_func),
    # Evaluate the check constraints
    migrations.RunSQL('SET CONSTRAINTS ALL IMMEDIATE;'),
    migrations.RemoveField(...). # Run the rest of your table operations
]

Running 'SET CONSTRAINTS ALL IMMEDIATE;' after the data migration ensures that all of those check constraints that normally wait until the end of the transaction will fire, meaning there are no more pending trigger events before the next ALTER TABLE statements.

FYI - This setting is effective for only the Django migration runner in that transaction. It won't affect other database sessions or any subsequent migration files.

Unlike the other answer linked, I'd avoid running an additional migrations.RunSQL('SET CONSTRAINTS ALL DEFERRED;'), otherwise you may alter the behavior of other triggers in subsequent steps that should run immediately (i.e. application triggers that track history, etc, that aren't check constraints).

Just wanted to add some more clarity here on exactly why this is happening since this is an older question with many different answers.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.