Django 1.9 release notes¶

Performing actions after a transaction commit¶

The new on_commit() hook allows performing actions after a database transaction is successfully committed. This is useful for tasks such as sending notification emails, creating queued tasks, or invalidating caches.

This functionality from the django-transaction-hooks package has been integrated into Django.

Password validation¶

Django now offers password validation to help prevent the usage of weak passwords by users. The validation is integrated in the included password change and reset forms and is simple to integrate in any other code. Validation is performed by one or more validators, configured in the new AUTH_PASSWORD_VALIDATORS setting.

Four validators are included in Django, which can enforce a minimum length, compare the password to the user’s attributes like their name, ensure passwords aren’t entirely numeric, or check against an included list of common passwords. You can combine multiple validators, and some validators have custom configuration options. For example, you can choose to provide a custom list of common passwords. Each validator provides a help text to explain its requirements to the user.

By default, no validation is performed and all passwords are accepted, so if you don’t set AUTH_PASSWORD_VALIDATORS, you will not see any change. In new projects created with the default startproject template, a simple set of validators is enabled. To enable basic validation in the included auth forms for your project, you could set, for example:

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

See Password validation for more details.

Permission mixins for class-based views¶

Django now ships with the mixins AccessMixin, LoginRequiredMixin, PermissionRequiredMixin, and UserPassesTestMixin to provide the functionality of the django.contrib.auth.decorators for class-based views. These mixins have been taken from, or are at least inspired by, the django-braces project.

There are a few differences between Django’s and django-braces” implementation, though:

• The raise_exception attribute can only be True or False. Custom exceptions or callables are not supported.
• The handle_no_permission() method does not take a request argument. The current request is available in self.request.
• The custom test_func() of UserPassesTestMixin does not take a user argument. The current user is available in self.request.user.
• The permission_required attribute supports a string (defining one permission) or a list/tuple of strings (defining multiple permissions) that need to be fulfilled to grant access.
• The new permission_denied_message attribute allows passing a message to the PermissionDenied exception.

New styling for contrib.admin¶

The admin sports a modern, flat design with new SVG icons which look perfect on HiDPI screens. It still provides a fully-functional experience to YUI’s A-grade browsers. Older browser may experience varying levels of graceful degradation.

Running tests in parallel¶

The test command now supports a --parallel option to run a project’s tests in multiple processes in parallel.

Each process gets its own database. You must ensure that different test cases don’t access the same resources. For instance, test cases that touch the filesystem should create a temporary directory for their own use.

This option is enabled by default for Django’s own test suite provided:

• the OS supports it (all but Windows)
• the database backend supports it (all the built-in backends but Oracle)

Minor features¶

Database backend API¶

• A couple of new tests rely on the ability of the backend to introspect column defaults (returning the result as Field.default). You can set the can_introspect_default database feature to False if your backend doesn’t implement this. You may want to review the implementation on the backends that Django includes for reference (#24245).

• Registering a global adapter or converter at the level of the DB-API module to handle time zone information of datetime values passed as query parameters or returned as query results on databases that don’t support time zones is discouraged. It can conflict with other libraries.

  The recommended way to add a time zone to datetime values fetched from the database is to register a converter for DateTimeField in DatabaseOperations.get_db_converters().

  The needs_datetime_string_cast database feature was removed. Database backends that set it must register a converter instead, as explained above.

• The DatabaseOperations.value_to_db_<type>() methods were renamed to adapt_<type>field_value() to mirror the convert_<type>field_value() methods.

• To use the new date lookup, third-party database backends may need to implement the DatabaseOperations.datetime_cast_date_sql() method.

• The DatabaseOperations.time_extract_sql() method was added. It calls the existing date_extract_sql() method. This method is overridden by the SQLite backend to add time lookups (hour, minute, second) to TimeField, and may be needed by third-party database backends.

• The DatabaseOperations.datetime_cast_sql() method (not to be confused with DatabaseOperations.datetime_cast_date_sql() mentioned above) has been removed. This method served to format dates on Oracle long before 1.0, but hasn’t been overridden by any core backend in years and hasn’t been called anywhere in Django’s code or tests.

• In order to support test parallelization, you must implement the DatabaseCreation._clone_test_db() method and set DatabaseFeatures.can_clone_databases = True. You may have to adjust DatabaseCreation.get_test_db_clone_settings().

Default settings that were tuples are now lists¶

The default settings in django.conf.global_settings were a combination of lists and tuples. All settings that were formerly tuples are now lists.

is_usable attribute on template loaders is removed¶

Django template loaders previously required an is_usable attribute to be defined. If a loader was configured in the template settings and this attribute was False, the loader would be silently ignored. In practice, this was only used by the egg loader to detect if setuptools was installed. The is_usable attribute is now removed and the egg loader instead fails at runtime if setuptools is not installed.

Related set direct assignment¶

Direct assignment of related objects in the ORM used to perform a clear() followed by a call to add(). This caused needlessly large data changes and prevented using the m2m_changed signal to track individual changes in many-to-many relations.

Direct assignment now relies on the new set() method on related managers which by default only processes changes between the existing related set and the one that’s newly assigned. The previous behavior can be restored by replacing direct assignment by a call to set() with the keyword argument clear=True.

ModelForm, and therefore ModelAdmin, internally rely on direct assignment for many-to-many relations and as a consequence now use the new behavior.

Filesystem-based template loaders catch more specific exceptions¶

When using the filesystem.Loader or app_directories.Loader template loaders, earlier versions of Django raised a TemplateDoesNotExist error if a template source existed but was unreadable. This could happen under many circumstances, such as if Django didn’t have permissions to open the file, or if the template source was a directory. Now, Django only silences the exception if the template source does not exist. All other situations result in the original IOError being raised.

HTTP redirects no longer forced to absolute URIs¶

Relative redirects are no longer converted to absolute URIs. RFC 2616 required the Location header in redirect responses to be an absolute URI, but it has been superseded by RFC 7231 which allows relative URIs in Location, recognizing the actual practice of user agents, almost all of which support them.

Consequently, the expected URLs passed to assertRedirects should generally no longer include the scheme and domain part of the URLs. For example, self.assertRedirects(response, 'http://testserver/some-url/') should be replaced by self.assertRedirects(response, '/some-url/') (unless the redirection specifically contained an absolute URL).

In the rare case that you need the old behavior (discovered with an ancient version of Apache with mod_scgi that interprets a relative redirect as an «internal redirect»), you can restore it by writing a custom middleware:

class LocationHeaderFix(object):
    def process_response(self, request, response):
        if 'Location' in response:
            response['Location'] = request.build_absolute_uri(response['Location'])
        return response

Dropped support for PostgreSQL 9.0¶

Upstream support for PostgreSQL 9.0 ended in September 2015. As a consequence, Django 1.9 sets 9.1 as the minimum PostgreSQL version it officially supports.

Dropped support for Oracle 11.1¶

Upstream support for Oracle 11.1 ended in August 2015. As a consequence, Django 1.9 sets 11.2 as the minimum Oracle version it officially supports.

Bulk behavior of add() method of related managers¶

To improve performance, the add() methods of the related managers created by ForeignKey and GenericForeignKey changed from a series of Model.save() calls to a single QuerySet.update() call. The change means that pre_save and post_save signals aren’t sent anymore. You can use the bulk=False keyword argument to revert to the previous behavior.

Template LoaderOrigin and StringOrigin are removed¶

In previous versions of Django, when a template engine was initialized with debug as True, an instance of django.template.loader.LoaderOrigin or django.template.base.StringOrigin was set as the origin attribute on the template object. These classes have been combined into Origin and is now always set regardless of the engine debug setting. For a minimal level of backwards compatibility, the old class names will be kept as aliases to the new Origin class until Django 2.0.

Changes to the default logging configuration¶

To make it easier to write custom logging configurations, Django’s default logging configuration no longer defines django.request and django.security loggers. Instead, it defines a single django logger, filtered at the INFO level, with two handlers:

• console: filtered at the INFO level and only active if DEBUG=True.
• mail_admins: filtered at the ERROR level and only active if DEBUG=False.

If you aren’t overriding Django’s default logging, you should see minimal changes in behavior, but you might see some new logging to the runserver console, for example.

If you are overriding Django’s default logging, you should check to see how your configuration merges with the new defaults.

HttpRequest details in error reporting¶

It was redundant to display the full details of the HttpRequest each time it appeared as a stack frame variable in the HTML version of the debug page and error email. Thus, the HTTP request will now display the same standard representation as other variables (repr(request)). As a result, the ExceptionReporterFilter.get_request_repr() method and the undocumented django.http.build_request_repr() function were removed.

The contents of the text version of the email were modified to provide a traceback of the same structure as in the case of AJAX requests. The traceback details are rendered by the ExceptionReporter.get_traceback_text() method.

Removal of time zone aware global adapters and converters for datetimes¶

Django no longer registers global adapters and converters for managing time zone information on datetime values sent to the database as query parameters or read from the database in query results. This change affects projects that meet all the following conditions:

• The USE_TZ setting is True.
• The database is SQLite, MySQL, Oracle, or a third-party database that doesn’t support time zones. In doubt, you can check the value of connection.features.supports_timezones.
• The code queries the database outside of the ORM, typically with cursor.execute(sql, params).

If you’re passing aware datetime parameters to such queries, you should turn them into naive datetimes in UTC:

from django.utils import timezone
param = timezone.make_naive(param, timezone.utc)

If you fail to do so, the conversion will be performed as in earlier versions (with a deprecation warning) up until Django 1.11. Django 2.0 won’t perform any conversion, which may result in data corruption.

If you’re reading datetime values from the results, they will be naive instead of aware. You can compensate as follows:

from django.utils import timezone
value = timezone.make_aware(value, timezone.utc)

You don’t need any of this if you’re querying the database through the ORM, even if you’re using raw() queries. The ORM takes care of managing time zone information.

Template tag modules are imported when templates are configured¶

The DjangoTemplates backend now performs discovery on installed template tag modules when instantiated. This update enables libraries to be provided explicitly via the 'libraries' key of OPTIONS when defining a DjangoTemplates backend. Import or syntax errors in template tag modules now fail early at instantiation time rather than when a template with a {% load %} tag is first compiled.

django.template.base.add_to_builtins() is removed¶

Although it was a private API, projects commonly used add_to_builtins() to make template tags and filters available without using the {% load %} tag. This API has been formalized. Projects should now define built-in libraries via the 'builtins' key of OPTIONS when defining a DjangoTemplates backend.

simple_tag now wraps tag output in conditional_escape¶

In general, template tags do not autoescape their contents, and this behavior is documented. For tags like inclusion_tag, this is not a problem because the included template will perform autoescaping. For assignment_tag(), the output will be escaped when it is used as a variable in the template.

For the intended use cases of simple_tag, however, it is very easy to end up with incorrect HTML and possibly an XSS exploit. For example:

@register.simple_tag(takes_context=True)
def greeting(context):
    return "Hello {0}!".format(context['request'].user.first_name)

In older versions of Django, this will be an XSS issue because user.first_name is not escaped.

In Django 1.9, this is fixed: if the template context has autoescape=True set (the default), then simple_tag will wrap the output of the tag function with conditional_escape().

To fix your simple_tags, it is best to apply the following practices:

• Any code that generates HTML should use either the template system or format_html().
• If the output of a simple_tag needs escaping, use escape() or conditional_escape().
• If you are absolutely certain that you are outputting HTML from a trusted source (e.g. a CMS field that stores HTML entered by admins), you can mark it as such using mark_safe().

Tags that follow these rules will be correct and safe whether they are run on Django 1.9+ or earlier.

Paginator.page_range¶

Paginator.page_range is now an iterator instead of a list.

In versions of Django previous to 1.8, Paginator.page_range returned a list in Python 2 and a range in Python 3. Django 1.8 consistently returned a list, but an iterator is more efficient.

Existing code that depends on list specific features, such as indexing, can be ported by converting the iterator into a list using list().

Implicit QuerySet __in lookup removed¶

In earlier versions, queries such as:

Model.objects.filter(related_id=RelatedModel.objects.all())

would implicitly convert to:

Model.objects.filter(related_id__in=RelatedModel.objects.all())

resulting in SQL like "related_id IN (SELECT id FROM ...)".

This implicit __in no longer happens so the «IN» SQL is now «=», and if the subquery returns multiple results, at least some databases will throw an error.

contrib.admin browser support¶

The admin no longer supports Internet Explorer 8 and below, as these browsers have reached end-of-life.

CSS and images to support Internet Explorer 6 and 7 have been removed. PNG and GIF icons have been replaced with SVG icons, which are not supported by Internet Explorer 8 and earlier.

The jQuery library embedded in the admin has been upgraded from version 1.11.2 to 2.1.4. jQuery 2.x has the same API as jQuery 1.x, but does not support Internet Explorer 6, 7, or 8, allowing for better performance and a smaller file size. If you need to support IE8 and must also use the latest version of Django, you can override the admin’s copy of jQuery with your own by creating a Django application with this structure:

app/static/admin/js/vendor/
    jquery.js
    jquery.min.js

SyntaxError when installing Django setuptools 5.5.x¶

When installing Django 1.9 or 1.9.1 with setuptools 5.5.x, you’ll see:

Compiling django/conf/app_template/apps.py ...
  File "django/conf/app_template/apps.py", line 4
    class {{ camel_case_app_name }}Config(AppConfig):
          ^
SyntaxError: invalid syntax

Compiling django/conf/app_template/models.py ...
  File "django/conf/app_template/models.py", line 1
    {{ unicode_literals }}from django.db import models
                             ^
SyntaxError: invalid syntax

It’s safe to ignore these errors (Django will still install just fine), but you can avoid them by upgrading setuptools to a more recent version. If you’re using pip, you can upgrade pip using python -m pip install -U pip which will also upgrade setuptools. This is resolved in later versions of Django as described in the Django 1.9.2 release notes.

Miscellaneous¶

• The jQuery static files in contrib.admin have been moved into a vendor/jquery subdirectory.
• The text displayed for null columns in the admin changelist list_display cells has changed from (None) (or its translated equivalent) to - (a dash).
• django.http.responses.REASON_PHRASES and django.core.handlers.wsgi.STATUS_CODE_TEXT have been removed. Use Python’s stdlib instead: http.client.responses for Python 3 and httplib.responses for Python 2.
• ValuesQuerySet and ValuesListQuerySet have been removed.
• The admin/base.html template no longer sets window.__admin_media_prefix__ or window.__admin_utc_offset__. Image references in JavaScript that used that value to construct absolute URLs have been moved to CSS for easier customization. The UTC offset is stored on a data attribute of the <body> tag.
• CommaSeparatedIntegerField validation has been refined to forbid values like ',', ',1', and '1,,2'.
• Form initialization was moved from the ProcessFormView.get() method to the new FormMixin.get_context_data() method. This may be backwards incompatible if you have overridden the get_context_data() method without calling super().
• Support for PostGIS 1.5 has been dropped.
• The django.contrib.sites.models.Site.domain field was changed to be unique.
• In order to enforce test isolation, database queries are not allowed by default in SimpleTestCase tests anymore. You can disable this behavior by setting the allow_database_queries class attribute to True on your test class.
• ResolverMatch.app_name was changed to contain the full namespace path in the case of nested namespaces. For consistency with ResolverMatch.namespace, the empty value is now an empty string instead of None.
• For security hardening, session keys must be at least 8 characters.
• Private function django.utils.functional.total_ordering() has been removed. It contained a workaround for a functools.total_ordering() bug in Python versions older than 2.7.3.
• XML serialization (either through dumpdata or the syndication framework) used to output any characters it received. Now if the content to be serialized contains any control characters not allowed in the XML 1.0 standard, the serialization will fail with a ValueError.
• CharField now strips input of leading and trailing whitespace by default. This can be disabled by setting the new strip argument to False.
• Template text that is translated and uses two or more consecutive percent signs, e.g. "%%", may have a new msgid after makemessages is run (most likely the translation will be marked fuzzy). The new msgid will be marked "#, python-format".
• If neither request.current_app nor Context.current_app are set, the url template tag will now use the namespace of the current request. Set request.current_app to None if you don’t want to use a namespace hint.
• The SILENCED_SYSTEM_CHECKS setting now silences messages of all levels. Previously, messages of ERROR level or higher were printed to the console.
• The FlatPage.enable_comments field is removed from the FlatPageAdmin as it’s unused by the application. If your project or a third-party app makes use of it, create a custom ModelAdmin to add it back.
• The return value of setup_databases() and the first argument of teardown_databases() changed. They used to be (old_names, mirrors) tuples. Now they’re just the first item, old_names.
• By default LiveServerTestCase attempts to find an available port in the 8081-8179 range instead of just trying port 8081.
• The system checks for ModelAdmin now check instances rather than classes.
• The private API to apply mixed migration plans has been dropped for performance reasons. Mixed plans consist of a list of migrations where some are being applied and others are being unapplied.
• The related model object descriptor classes in django.db.models.fields.related (private API) are moved from the related module to related_descriptors and renamed as follows:
  • ReverseSingleRelatedObjectDescriptor is ForwardManyToOneDescriptor
  • SingleRelatedObjectDescriptor is ReverseOneToOneDescriptor
  • ForeignRelatedObjectsDescriptor is ReverseManyToOneDescriptor
  • ManyRelatedObjectsDescriptor is ManyToManyDescriptor
• If you implement a custom handler404 view, it must return a response with an HTTP 404 status code. Use HttpResponseNotFound or pass status=404 to the HttpResponse. Otherwise, APPEND_SLASH won’t work correctly with DEBUG=False.

assignment_tag()¶

Django 1.4 added the assignment_tag helper to ease the creation of template tags that store results in a template variable. The simple_tag() helper has gained this same ability, making the assignment_tag obsolete. Tags that use assignment_tag should be updated to use simple_tag.

{% cycle %} syntax with comma-separated arguments¶

The cycle tag supports an inferior old syntax from previous Django versions:

{% cycle row1,row2,row3 %}

Its parsing caused bugs with the current syntax, so support for the old syntax will be removed in Django 1.10 following an accelerated deprecation.

ForeignKey and OneToOneField on_delete argument¶

In order to increase awareness about cascading model deletion, the on_delete argument of ForeignKey and OneToOneField will be required in Django 2.0.

Update models and existing migrations to explicitly set the argument. Since the default is models.CASCADE, add on_delete=models.CASCADE to all ForeignKey and OneToOneFields that don’t use a different option. You can also pass it as the second positional argument if you don’t care about compatibility with older versions of Django.

Field.rel changes¶

Field.rel and its methods and attributes have changed to match the related fields API. The Field.rel attribute is renamed to remote_field and many of its methods and attributes are either changed or renamed.

The aim of these changes is to provide a documented API for relation fields.

GeoManager and GeoQuerySet custom methods¶

All custom GeoQuerySet methods (area(), distance(), gml(), …) have been replaced by equivalent geographic expressions in annotations (see in new features). Hence the need to set a custom GeoManager to GIS-enabled models is now obsolete. As soon as your code doesn’t call any of the deprecated methods, you can simply remove the objects = GeoManager() lines from your models.

Template loader APIs have changed¶

Django template loaders have been updated to allow recursive template extending. This change necessitated a new template loader API. The old load_template() and load_template_sources() methods are now deprecated. Details about the new API can be found in the template loader documentation.

Passing a 3-tuple or an app_name to include()¶

The instance namespace part of passing a tuple as an argument to include() has been replaced by passing the namespace argument to include(). For example:

polls_patterns = [
     url(...),
]

urlpatterns = [
    url(r'^polls/', include((polls_patterns, 'polls', 'author-polls'))),
]

becomes:

polls_patterns = ([
     url(...),
], 'polls')  # 'polls' is the app_name

urlpatterns = [
    url(r'^polls/', include(polls_patterns, namespace='author-polls')),
]

The app_name argument to include() has been replaced by passing a 2-tuple (as above), or passing an object or module with an app_name attribute (as below). If the app_name is set in this new way, the namespace argument is no longer required. It will default to the value of app_name. For example, the URL patterns in the tutorial are changed from:

urlpatterns = [
    url(r'^polls/', include('polls.urls', namespace="polls")),
    ...
]

to:

urlpatterns = [
    url(r'^polls/', include('polls.urls')),  # 'namespace="polls"' removed
    ...
]

app_name = 'polls'  # added
urlpatterns = [...]

This change also means that the old way of including an AdminSite instance is deprecated. Instead, pass admin.site.urls directly to django.conf.urls.url():

from django.conf.urls import url
from django.contrib import admin

urlpatterns = [
    url(r'^admin/', admin.site.urls),
]

URL application namespace required if setting an instance namespace¶

In the past, an instance namespace without an application namespace would serve the same purpose as the application namespace, but it was impossible to reverse the patterns if there was an application namespace with the same name. Includes that specify an instance namespace require that the included URLconf sets an application namespace.

current_app parameter to contrib.auth views¶

All views in django.contrib.auth.views have the following structure:

def view(request, ..., current_app=None, ...):

    ...

    if current_app is not None:
        request.current_app = current_app

    return TemplateResponse(request, template_name, context)

As of Django 1.8, current_app is set on the request object. For consistency, these views will require the caller to set current_app on the request instead of passing it in a separate argument.

django.contrib.gis.geoip¶

The django.contrib.gis.geoip2 module supersedes django.contrib.gis.geoip. The new module provides a similar API except that it doesn’t provide the legacy GeoIP-Python API compatibility methods.

Miscellaneous¶

• The weak argument to django.dispatch.signals.Signal.disconnect() has been deprecated as it has no effect.
• The check_aggregate_support() method of django.db.backends.base.BaseDatabaseOperations has been deprecated and will be removed in Django 2.0. The more general check_expression_support() should be used instead.
• django.forms.extras is deprecated. You can find SelectDateWidget in django.forms.widgets (or simply django.forms) instead.
• Private API django.db.models.fields.add_lazy_relation() is deprecated.
• The django.contrib.auth.tests.utils.skipIfCustomUser() decorator is deprecated. With the test discovery changes in Django 1.6, the tests for django.contrib apps are no longer run as part of the user’s project. Therefore, the @skipIfCustomUser decorator is no longer needed to decorate tests in django.contrib.auth.
• If you customized some error handlers, the view signatures with only one request parameter are deprecated. The views should now also accept a second exception positional parameter.
• The django.utils.feedgenerator.Atom1Feed.mime_type and django.utils.feedgenerator.RssFeed.mime_type attributes are deprecated in favor of content_type.
• Signer now issues a warning if an invalid separator is used. This will become an exception in Django 1.10.
• django.db.models.Field._get_val_from_obj() is deprecated in favor of Field.value_from_object().
• django.template.loaders.eggs.Loader is deprecated as distributing applications as eggs is not recommended.
• The callable_obj keyword argument to SimpleTestCase.assertRaisesMessage() is deprecated. Pass the callable as a positional argument instead.
• The allow_tags attribute on methods of ModelAdmin has been deprecated. Use format_html(), format_html_join(), or mark_safe() when constructing the method’s return value instead.
• The enclosure keyword argument to SyndicationFeed.add_item() is deprecated. Use the new enclosures argument which accepts a list of Enclosure objects instead of a single one.
• The django.template.loader.LoaderOrigin and django.template.base.StringOrigin aliases for django.template.base.Origin are deprecated.