Out of curiosity, what's the reason the example index can't be defined as

Concat(Lower(F('name')), Value('blub'))

@charettes Apparently, I haven't fully understood it, F() looks up fields in the Compiler and refers to already mentioned fields. However, the Compiler used here doesn't have any mentioned fields, thus using F() fails. But that's something I want to get to: Concat(Lower('name'), Value('blub'))

It's been a little while since I looked into this, but you can't use F() because:

1. F doesn't define an as_sql() method because;
2. F must be resolved by the query, which turns it into either a Col or a Ref (I suspect a Col would work as well), which do have as_sql() methods
3. To resolve an F, you need to hand it a query (+ joins), and the query is used to setup joins if needed.

You might be able to work around this by passing F.resolve_expression a fake query object with a resolve_ref method that simply verifies the column existing on the model, and returning the appropriate Col or Ref. Col looks like it outputs "table.column" while Ref outputs "column_alias" which can just be the column name.

You could try building non-query support into the F expression by overloading the resolve_expression method, but that kind of defeats the purpose of using F anyway, while complicating the code. F doesn't do the column/reference verification, that's handled by query.resolve_ref.

Hope that helps. Also, nice work! I really look forward to seeing this available.

This is a second shot at how function based indexes could look like.

$ cat app/models.py

from django.db import models
from django.db.models.functions import Lower


class Foo(models.Model):
    name = models.CharField(max_length=255)

    class Meta:
        indexes = [
            models.Index(
                [Lower('name').desc()],
                name='some_func_index'
            ),
        ]

$ python manage.py makemigrations -v 3
Migrations for 'app':
  app/migrations/0001_initial.py
    - Create model Foo
    - Create index some_func_index for model foo

$ cat app/migrations/0001_initial.py

# Generated by Django 2.0.dev20170507133517 on 2017-05-07 13:52

from django.db import migrations, models
import django.db.models.expressions
import django.db.models.functions.base


class Migration(migrations.Migration):

    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Foo',
            fields=[
                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
                ('name', models.CharField(max_length=255)),
            ],
        ),
        migrations.AddIndex(
            model_name='foo',
            index=models.Index(fields=[django.db.models.expressions.OrderBy(django.db.models.functions.base.Lower('name'), descending=True)], name='some_func_index'),
        ),
    ]

On SQLite3

$ python manage.py sqlmigrate app 0001

BEGIN;
--
-- Create model Foo
--
CREATE TABLE "app_foo" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "name" varchar(255) NOT NULL);
--
-- Create index some_func_index on field(s) OrderBy(Lower(F(name)), descending=True) of model foo
--
CREATE INDEX "some_func_index" ON "app_foo" (LOWER("name") DESC);
COMMIT;

$ python manage.py migrate app
Operations to perform:
  Apply all migrations: app
Running migrations:
  Applying app.0001_initial... OK

$ python manage.py dbshell

SQLite version 3.16.2 2017-01-06 16:32:41
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE IF NOT EXISTS "django_migrations" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "app" varchar(255) NOT NULL, "name" varchar(255) NOT NULL, "applied" datetime NOT NULL);
CREATE TABLE IF NOT EXISTS "app_foo" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "name" varchar(255) NOT NULL);
CREATE INDEX "some_func_index" ON "app_foo" (LOWER("name") DESC);
sqlite> .quit

On PostgreSQL

$ python manage.py sqlmigrate app 0001 --settings=testproject.pg_settings

BEGIN;
--
-- Create model Foo
--
CREATE TABLE "app_foo" ("id" serial NOT NULL PRIMARY KEY, "name" varchar(255) NOT NULL);
--
-- Create index some_func_index on field(s) OrderBy(Lower(F(name)), descending=True) of model foo
--
CREATE INDEX "some_func_index" ON "app_foo" (LOWER("name") DESC);
COMMIT;

$ python manage.py migrate app --settings=testproject.pg_settings
Operations to perform:
  Apply all migrations: app
Running migrations:
  Applying app.0001_initial... OK

$ python manage.py dbshell --settings=testproject.pg_settings

psql (9.6.2)
Type "help" for help.

django=> \dS+ app1_foo
                                                     Table "public.app1_foo"
 Column |          Type          |                       Modifiers                       | Storage  | Stats target | Description 
--------+------------------------+-------------------------------------------------------+----------+--------------+-------------
 id     | integer                | not null default nextval('app1_foo_id_seq'::regclass) | plain    |              | 
 name   | character varying(255) | not null                                              | extended |              | 
Indexes:
    "app1_foo_pkey" PRIMARY KEY, btree (id)
    "some_func_index" btree (lower(name::text) DESC)

I found this error:

======================================================================
ERROR: test_func_index (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/tests/schema/tests.py", line 1771, in test_func_index
    sql = func_index.create_sql(Book, editor)
  File "/django/django/db/models/indexes.py", line 78, in create_sql
    sql_parameters = self.get_sql_create_template_values(model, schema_editor, using)
  File "/django/django/db/models/indexes.py", line 58, in get_sql_create_template_values
    expression = column_expression.resolve_expression(query)
  File "/django/django/db/models/functions/datetime.py", line 60, in resolve_expression
    field = copy.lhs.output_field
  File "/django/django/utils/functional.py", line 31, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/django/django/db/models/expressions.py", line 226, in output_field
    raise FieldError("Cannot resolve expression type, unknown output_field")
django.core.exceptions.FieldError: Cannot resolve expression type, unknown output_field

when I had added index with ExtractMonth to the test_func_index:

func_index = Index(fields=[ExtractMonth('pub_date')], name='pub_data_month_idx')
with connection.schema_editor() as editor:
    sql = func_index.create_sql(Book, editor)

Thanks @apollo13 @felixxm . I think I took care of both your comments. The tests do need some major improvements.

Thanks @felixxm. Updated.

@MarkusH I had a look in get_constraints and in PostgreSQL's backend the expression definition is part of this. Would it be an idea to include tests for this? The SQL for the index is only included in cases like this where you use an expression. However, a cast may be added to the SQL, e.g., lower(column) is lower(column::text) if column is not of type text. Oracle probably has something similar to pg_get_indexdef. SQLite doesn't seem like it does.

Please do rebase.

Would be nice to have a test case using a subclass of Index, just to ensure it works. Example (IIUC):

from django.db import models, Func, F
from django.contrib.postgres.indexes import GistIndex


class Location(models.Model):
    lat = models.FloatField()
    lon = models.FloatField()

    class Meta:
        indexes = [
            GistIndex(fields=[Func(F('lat'), F('lon'), function='ll_to_earth')]),
        ]

should result in

CREATE INDEX idx_name ON app_location USING gist(ll_to_earth("lat", "lng"));

I've been working on a CalculatedField, which has a significant overlap with this (in the sense of that the only part of my field that does not work is when you try to create an index on it).

https://github.com/schinckel/django-computed-field

Ignoring the fact I have to currently sniff up the stack to resolve the query (I'm thinking about trying to use Ref instead of F though, based on reading the stuff here), I wonder if this is a solution to this problem that could be considered. You could (hopefully) just use the computed field by name in the index stuff, and then it would handle all of the expression handling.

Any updates?

@MarkusH this would be great to have in, any plans on getting back to it?

Superseded by #11929