28
loading...
Understand Django: Command Your App
In the last Understand Django article, we dug into file management. We saw how Django handles user uploaded files and how to deal with them safely.
With this article, you'll learn about commands. Commands are the way to execute scripts that interact with your Django app. We'll see built-in commands and how to build your own.
Django makes it possible to run code from a terminal with
./manage.py, but why is this helpful or needed? Consider this script:# product_stat.py
from application.models import Product
print(Product.objects.count())which you could try running with:
$ python product_stat.pyThe problem with this script is that Django is not ready to run yet. If you tried to run this kind of code, you would get an
ImproperlyConfigured exception. There are a couple of modifications that you could make to get the script to run.django.setup().DJANGO_SETTINGS_MODULE.
# product_stat.py
import django
django.setup()
from application.models import Product
print(Product.objects.count())Note that
django.setup() must be before your Django related imports (like the Product model in this example). Now the script can run if you supply where the settings are located too.$ DJANGO_SETTING_MODULE=project.settings python product_stat.pyThis arrangement is less than ideal, but why else might we want a way to run commands through Django?
Try running
./manage.py -h.What you'll likely see is more commands than what Django provides alone. This is where we begin to see more value from the command system. Because Django provides a standard way to run scripts, other Django applications can bundle useful commands and make them easily accessible to you.
Now that you've had a chance to see why commands exist to run scripts for Django apps, let's back up and see what commands are.
Django gives us a tool to run commands before we've even started our project. That tool is the
django-admin script. We saw it all the way back in the first article where I provided a short set of setup instructions to get you started if you've never used Django before.After you've started a project, your code will have a
manage.py file, and the commands you've seen in most articles are in the form of:$ ./manage.py some_commandWhat's the difference between
django-admin and manage.py? In truth, not much!django-admin comes from Django's Python packaging. In Python packages, package developers can create scripts by defining an entry point in the packaging configuration. In Django, this configuration looks like:[options.entry_points]
console_scripts =
django-admin = django.core.management:execute_from_command_lineMeanwhile, the entire
manage.py of one of my projects is:#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys
def main():
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
if __name__ == '__main__':
main()If you look closely, you can see that the different scripts are both ways to invoke the
execute_from_command_line function to run a command. The primary difference is that the latter script will attempt to set the DJANGO_SETTINGS_MODULE environment variable automatically. Since Django needs to have DJANGO_SETTINGS_MODULE defined for most commands (note: startproject does not require that variable), manage.py is a more convenient way to run commands.execute_from_command_line is able to present what commands are available to a project, whether a command comes from Django itself, an installed app, or is a custom command that you created yourself. How are the commands discovered? The command system does discovery by following some packaging conventions.Let's say your project has an app named
application. Django can find the command if you have the following packaging structure.application
├── __init__.py
├── management
│ ├── __init__.py
│ └── commands
│ ├── __init__.py
│ └── custom_command.py
├── models.py
└── views.py
... Other typical Django app filesWith this structure, you could run:
$ ./manage.py custom_commandNotes:
<app>/management/commands/<command name>.py.__init__.py files! Django can only discover the commands if management and commands are proper Python package directories.custom_command, but you can name your command with whatever valid Python module name that you want.Unfortunately, we can't slap some Python code into
custom_command.py and assume that Django will know how to run it. Within the custom_command.py module, Django needs to find a Command class that subclasses a BaseCommand class that is provided by the framework. Django requires this structure to give command authors a consistent way to access features of the command system.With the
Command class, you can add a help class attribute. Adding help can gives users a description of what your command does when running ./manage.py custom_command -h.The
Command class will also help you with handling arguments. If your command needs to work with user input, you'll need to parse that data. Thankfully, the class integrates with Python's built-in argparse module. By including an add_arguments method, a command can parse the data and pass the results to the command's handler method in a structured way. If you've had to write Python scripts before, then you may understand how much time this kind of parsing can save you (and for those who haven't, the answer is "a lot of time!").Other smaller features exist within the
Command class too. Perhaps you only want your command to run if your project has satisified certain pre-conditions. Commands can use the requires_migration_checks or requires_system_checks to ensure that the system is in the correct state before running.I hope it's clear that the goal of the
Command class is to help you with common actions that many commands will need to use. There is a small API to learn, but the system is a boon to making scripts quickly.Let's consider a powerful use case to see a command in action. When you initially start a Django app, all of your app's interaction will probably be through web pages. After all, you were trying to use Django to make a web app, right? What do you when you need to do something that doesn't involve a browser?
This kind of work for your app is often considered background work. Background work is a pretty deep topic and will often involve special background task software like Celery. When your app is an early stage, Celery or similar software can be overkill and far more than you need.
A simpler alternative for some background tasks could be a command paired with a scheduling tool like cron.
On one of my projects, I offer free trials for accounts. After 60 days, the free trial ends and users either need to pay for the service or discontinue using it. By using a command and pairing it with the Heroku Scheduler, I can move accounts from their trial status to expired with a daily check.
The following code is very close to what my
expire_trials command looks like in my app. I've simplified things a bit, so that you can ignore the details that are specific to my service.# application/management/commands/expire_trials.py
import datetime
from django.core.management.base import BaseCommand
from django.utils import timezone
from application.models import Account
class Command(BaseCommand):
help = "Expire any accounts that are TRIALING beyond the trial days limit"
def handle(self, *args, **options):
self.stdout.write("Search for old trial accounts...")
# Give an extra day to be gracious and avoid customer complaints.
cutoff_days = 61
trial_cutoff = timezone.now() - datetime.timedelta(days=cutoff_days)
expired_trials = Account.objects.filter(
status=Account.TRIALING, created__lt=trial_cutoff
)
count = expired_trials.update(status=Account.TRIAL_EXPIRED)
self.stdout.write(f"Expired {count} trial(s)")I configured the scheduler to run
python manage.py expire_trials every day in the early morning. The command checks the current time and looks for Account records in the trialing state that were created before the cutoff time. From that query set, the affected accounts are set to the expired account state.How can you test this command? There are a couple of approaches you can take when testing a command.
If you need to simulate calling the command with command line arguments, then you can use
call_command from django.core.management. Since the example command doesn't require arguments, I didn't take that approach.Generally, my preference is to create a command object and invoke the
handle method directly. In my example above, you can see that the command uses self.stdout instead of calling print. Django does this so that you could check your output if desired.Here is a test for this command:
# application/tests/test_commands.py
from io import StringIO
from application.models import Account
from application.tests.factories import AccountFactory
def test_expires_trials():
"""Old trials are marked as expired."""
stdout = StringIO()
account = AccountFactory(
status=Account.TRIALING,
created=timezone.now() - datetime.timedelta(days=65),
)
command = Command(stdout=stdout)
command.handle()
account.refresh_from_db()
assert account.status == Account.TRIAL_EXPIRED
assert "Expired 1 trial(s)" in stdout.getvalue()In this test, I constructed a command instance and checked the account state after the command invocation. Also, observe that the
StringIO instance is injected into the Command constructor. By building the command this way, checking the output becomes a very achievable task via the getvalue method.Overall, this scheme of making a command and running it on a schedule avoids all the work of setting up a background worker process. I've been extremely satisfied with how this technique has worked for me, and I think it's a great pattern when your app doesn't have to do a lot of complex background processing.
Django is full of useful commands that you can use for all kinds of purposes. Thus far in this series, we've discussed a bunch of them, including:
check - Checks that your project is in good shape.collectstatic - Collects static files into a single directory.createsuperuser - Creates a super user record.makemigrations - Makes new migration files based on model changes.migrate - Runs any unapplied migrations to your database.runserver - Runs a development web server to check your app.shell - Starts a Django shell that allows you to use Django code on the command line.startapp - Makes a new Django app from a template.startproject - Makes a new Django project from a template.test - Executes tests that checks the validity of your app.Here is a sampling of other commands that I find useful when working with Django projects.
The
dbshell command starts a different kind of shell. The shell is a database program that will connect to the same database that your Django app uses. This shell will vary based on your choice of database.For instance, when using PostgreSQL,
./manage.py dbshell will start psql. From this shell, you can execute SQL statements directly to inspect the state of your database. I don't reach for this command often, but I find it very useful to connect to my database without having to remember database credentials.The
showmigrations command has a simple job. The command shows all the migrations for each Django app in your project. Next to each migration is an indicator of whether the migration is applied to your database.Here is an example of the
users app from one of my Django projects:$ ./manage.py showmigrations users
users
[X] 0001_initial
[X] 0002_first_name_to_150_max
[ ] 0003_profileIn my real project, I've applied all the migrations, but for this example, I'm showing the third migration as it would appear if the migration wasn't applied yet.
showmigrations is a good way to show the state of your database from Django's point of view.The
sqlmigrate command is very handy. The command will show you what SQL statements Django would run for an individual migration file.Let's see an example. In Django 3.1, the team changed the
AbstractUser model so that the first_name field could have a maximum length of 150 characters. Anyone using the AbstractUser model (which includes me) had to generate a migration to apply that change.From my
showmigrations output above, you can see that the second migration of my users app applied this particular framework change.To see the Postgres SQL statements that made the change, I can run:
$ ./manage.py sqlmigrate users 0002
BEGIN;
--
-- Alter field first_name on user
--
ALTER TABLE "users_user" ALTER COLUMN "first_name" TYPE varchar(150);
COMMIT;From this, we can tell that Postgres executed an
ALTER COLUMN DDL statement to modify the length of the first_name field.Django migrations are a stack of separate database changes that produce a final desired schema state in your database. Over time, your Django apps will accumulate migration files, but those files have a shelf life. The
squashmigrations command is designed to let you tidy up an app's set of migration files.By running
squashmigrations, you can condense an app's migrations into a significantly smaller number. The reduced migrations can accurately represent your database schema, and make it easier to reason about what changes happened in the app's history. As a side benefit, migration squashing can make Django's migration handling faster because Django gets to process fewer files.The commands above come with the standard Django install. There's even more cool stuff out there to help with your project development!
A package that I often reach for with my Django projects is the django-extensions package. This package is fully of goodies, including some great optional commands that you can use!
A couple of my favorites include:
How often do you fire up a Django shell, import a model, then do some ORM queries to see the current state of the database? This is something I do quite often.
The
shell_plus command is like the regular shell, but the command will import all your models automatically. For the five extra characters of _plus, you can save your fingers a lot of typing to import your models and get directly to whatever you needed the shell for.The command will also import some commonly used Django functions and features like
reverse, settings, timezone, and more.When I'm live streaming my side projects on my Twitch channel, I will often want to show the model relationships of my Django project. With the
graph_models command, I can create an image of all my models and how those models relate to each other (using UML syntax). This is a great way to:This particular command requires some extra setup to install the right tools to create images, but the setup is manageable and the results are worth it.
Aside from
shell_plus and graph_models, there are 20 other commands that you can use that may be very useful to you. You should definitely check out django-extensions.In this article, you saw Django commands. We covered:
In the next article, I'm going to describe internationalization. Internationalization is the process of making your app work for multiple spoken languages. You'll learn about:
If you'd like to follow along with the series, please feel free to sign up for my newsletter where I announce all of my new content. If you have other questions, you can reach me online on Twitter where I am @mblayman.
28