Skip to main content

Automatic Schema Migration Planning for Django

TL;DR​

  • Django is a Python web framework with a built-in ORM.
  • Atlas is an open-source tool for inspecting, planning, linting and executing schema changes to your database.
  • Developers using Django can use Atlas to automatically plan schema migrations for them.

Automatic Migration Planning for Django​

Django is the most popular web framework in the Python community developed in 2014. It includes a built-in ORM which allows users to describe their data model using Python classes.

You can create migrations with Django using the python manage.py makemigrations command and apply them to the database using python manage.py migrate.

Out of the many ORMs available, Django's automatic migration is one of the most powerful and robust. However, having been created in a very different era in software engineering, it naturally has some limitations:

  • Database Features. Created to provide interoperability across database engines, Django's migration system is centered around the "lowest common denominator" of database features. More advanced features such as Triggers, Views, and Stored Procedures have very limited support and require more work from developers to include in migrations.
  • Ensuring Migration Safety. Django's migration system does not provide a native way to ensure that a migration is safe to apply without leading to data loss or a production outage.
  • Modern Deployments. Django does not provide native integration with modern deployment practices such as GitOps or Infrastructure-as-Code.

Atlas lets you manage your Django applications using the Database Schema-as-Code paradigm so you can enjoy automatic migration planning, automatic code review, and integrations with your favorite CI/CD tools.

In this guide, we will show how Django users can use Atlas to automatically plan schema migrations.

The Atlas Django Provider​

In this guide, we will use the Django Atlas Provider to automatically plan schema migrations for a Django project.

Installation​

To download and install the latest release of the Atlas CLI, simply run the following in your terminal:

curl -sSf https://atlasgo.sh | sh

Start a Python virtual environment:

python3 -m venv venv
source venv/bin/activate

Install the provider by running:

pip install atlas-provider-django

Configuration​

Add the provider to your Django project's INSTALLED_APPS in settings.py:

INSTALLED_APPS = [
    ...,
    'atlas_provider_django',
    ...
]

In your project directory, where manage.py is located, create a new file named atlas.hcl with the following contents:

data "external_schema" "django" {
  program = [
    "python",
    "manage.py",
    "atlas-provider-django",
    "--dialect", "mysql" // mariadb | postgresql | sqlite | mssql
    
    // if you want to only load a subset of your app models, you can specify the apps by adding
    // "--apps", "app1", "app2", "app3"
  ]
}

env "django" {
  src = data.external_schema.django.url
  dev = "docker://mysql/8/dev"
  migration {
    dir = "file://migrations"
  }
  format {
    migrate {
      diff = "{{ sql . \"  \" }}"
    }
  }
}

Versioned Migrations​

Atlas supports a versioned migrations workflow, where each change to the database is versioned and recorded in a migration file. The atlas migrate diff command automatically generates a migration file that will migrate the database from its latest revision to your current Django schema definition.

Create your first migration​

Suppose we have an empty database and an an app named polls with the following models in our polls/models.py file:

from django.db import models


class Question(models.Model):
    question_text = models.CharField(max_length=200)
    pub_date = models.DateTimeField("date published")


class Choice(models.Model):
    question = models.ForeignKey(Question, on_delete=models.CASCADE)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)

Generate a migration file by running this command:

atlas migrate diff --env django

This will result in a new migrations directory that looks similar to:

migrations
├── 20240126104629.sql
└── atlas.sum

0 directories, 2 files

Examining the contents of 20240126104629.sql:

-- Create "polls_question" table
CREATE TABLE `polls_question` (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `question_text` varchar(200) NOT NULL,
  `pub_date` datetime(6) NOT NULL,
  PRIMARY KEY (`id`)
) CHARSET utf8mb4 COLLATE utf8mb4_0900_ai_ci;
-- Create "polls_choice" table
CREATE TABLE `polls_choice` (
  `id` bigint NOT NULL AUTO_INCREMENT,
  `choice_text` varchar(200) NOT NULL,
  `votes` int NOT NULL,
  `question_id` bigint NOT NULL,
  PRIMARY KEY (`id`),
  INDEX `polls_choice_question_id_c5b4b260_fk_polls_question_id` (`question_id`),
  CONSTRAINT `polls_choice_question_id_c5b4b260_fk_polls_question_id` FOREIGN KEY (`question_id`) REFERENCES `polls_question` (`id`) ON UPDATE NO ACTION ON DELETE NO ACTION
) CHARSET utf8mb4 COLLATE utf8mb4_0900_ai_ci;

Amazing! Atlas automatically generated a migration file that will create the polls_question and polls_choice tables in our database.

Create another migration​

Now, let's alter the Question class in our models.py schema definition to add a new question_type field:

class Question(models.Model):
    question_text = models.CharField(max_length=200)
+   question_type = models.CharField(max_length=20, null=True)
    pub_date = models.DateTimeField("date published")

Re-run this command:

atlas migrate diff --env django

Observe a new migration file is generated:

-- Modify "polls_question" table
ALTER TABLE `polls_question` ADD COLUMN `question_type` varchar(20) NULL;

Conclusion​

In this guide we demonstrated how projects using Django can use Atlas to automatically plan schema migrations based on their data model. To learn more about applying these migrations to your production database, read our documentation for the migrate apply command.

Have questions? Feedback? Find our team on our Discord server