CodeOath
← All posts
Python65 min total · 18 parts

Django Fundamentals: The ORM, Migrations, and Shipping a Real App

Part 5 of 18 · ~4 min

Migrations: Priya and Theo Step on Each Other

python manage.py makemigrations   # write a migration file describing the model change
python manage.py migrate          # apply pending migrations to the database
python manage.py sqlmigrate loans 0002   # print the actual SQL a given migration will run
python manage.py showmigrations   # see what's applied vs. still pending

A migration is a small, reviewable Python file, checked into the same repository as everything else, that spells out one specific change to the database's shape — and crucially, every environment that ever runs the project replays the full stack of them in the exact same order, so Priya's laptop and whatever server ToolShed eventually lives on never drift out of sync with each other.

sqlmigrate is worth actually running before trusting what a migration does, rather than assuming from the Python. Adding one boolean field to Loan on SQLite — verified directly — doesn't come back as the single ALTER TABLE you'd expect:

BEGIN;
CREATE TABLE "new__loans_loan" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
    "due_reminder_sent" bool NOT NULL, "status" varchar(20) NOT NULL, ...);
INSERT INTO "new__loans_loan" ("id", "status", ..., "due_reminder_sent")
    SELECT "id", "status", ..., 0 FROM "loans_loan";
DROP TABLE "loans_loan";
ALTER TABLE "new__loans_loan" RENAME TO "loans_loan";
COMMIT;

SQLite can't add a NOT NULL column to an existing table in place, so Django works around that by building a new table with the extra column, copying every row across with the field's default filled in, and swapping the names — four statements standing in for what looks, from the model change alone, like it should be one. Postgres and MySQL handle the same AddField with a genuine single ALTER TABLE; the lesson isn't about SQLite specifically, it's that sqlmigrate is the only honest way to know what a migration is actually about to do to a real table, rather than guessing from the Python that generated it.

The Saturday two contributors collided

ToolShed gained a second contributor, Theo, a few months in — he wanted Tool to track how many times it had actually been loaned out, for a "popular tools" section. On the same Saturday, entirely by coincidence, Priya was adding a due-date reminder flag to Loan. Both of them ran makemigrations against the same starting point, both got a clean file, both committed. The moment Theo pulled Priya's branch:

$ python manage.py makemigrations
CommandError: Conflicting migrations detected; multiple leaf nodes
in the migration graph: (0003_tool_times_loaned, 0003_loan_due_reminder_sent in loans).
To fix them run 'python manage.py makemigrations --merge'

Nothing is actually broken. Django is reporting, accurately, that the migration history forked into two branches that both claim to follow migration 0002 — the exact same shape as two Git branches that both moved past a shared commit and now need a merge commit to reconcile. The fix is the command Django already suggested:

python manage.py makemigrations --merge

This generates a new migration whose only job is depending on both 0003 files at once, giving the graph a single leaf again. It doesn't touch either contributor's actual schema change — it just tells Django "both of these happened, in no particular order relative to each other, and here's where the history reconverges."

Backfilling times_loaned without lying about history

Theo's new field needs more than a bare AddField, though — it needs a real starting value for every tool that already had loans before the field existed, or every popular-tools ranking starts flat at zero regardless of actual history. That's a data migration, written with RunPython:

from django.db import migrations


def backfill_times_loaned(apps, schema_editor):
    # the historical model, frozen exactly as the schema looked at THIS point
    # in migration history — not the "live" Tool class from catalog/models.py,
    # which may already have fields this migration predates
    Tool = apps.get_model("catalog", "Tool")
    Loan = apps.get_model("loans", "Loan")
    for tool in Tool.objects.all():
        tool.times_loaned = Loan.objects.filter(tool=tool).count()
        tool.save()


class Migration(migrations.Migration):
    dependencies = [("catalog", "0003_tool_times_loaned")]
    operations = [migrations.RunPython(backfill_times_loaned)]

The reason for apps.get_model instead of the obvious from catalog.models import Tool shows up only much later, and only on a database that hasn't run every migration yet. Someone spinning up a brand-new environment replays every migration from the very first one in order, and a data migration sitting at position twelve is running against a Tool table as it looked at step twelve — not as it looks in today's catalog/models.py, which might have three more fields on it than existed back then. Import the live class and this migration quietly assumes those later fields were already there; apps.get_model hands back the version of Tool frozen at exactly this point in the sequence, which is the only version that migration is actually allowed to assume.