Improved Django model inheritance with automatic downcasting

Last update: Jan 3, 2023

Related tags

Overview

https://travis-ci.org/django-polymorphic/django-polymorphic.svg?branch=master

https://readthedocs.org/projects/django-polymorphic/badge/?version=stable

Polymorphic Models for Django

Django-polymorphic simplifies using inherited models in Django projects. When a query is made at the base model, the inherited model classes are returned.

When we store models that inherit from a Project model...

>>> Project.objects.create(topic="Department Party")
>>> ArtProject.objects.create(topic="Painting with Tim", artist="T. Turner")
>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")

...and want to retrieve all our projects, the subclassed models are returned!

>>> Project.objects.all()
[ <Project:         id 1, topic "Department Party">,
  <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner">,
  <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]

Using vanilla Django, we get the base class objects, which is rarely what we wanted:

>>> Project.objects.all()
[ <Project: id 1, topic "Department Party">,
  <Project: id 2, topic "Painting with Tim">,
  <Project: id 3, topic "Swallow Aerodynamics"> ]

This also works when the polymorphic model is accessed via ForeignKeys, ManyToManyFields or OneToOneFields.

Features

Full admin integration.
ORM integration:
- support for ForeignKey, ManyToManyField, OneToOneField descriptors.
- Filtering/ordering of inherited models (ArtProject___artist).
- Filtering model types: instance_of(...) and not_instance_of(...)
- Combining querysets of different models (qs3 = qs1 | qs2)
- Support for custom user-defined managers.
Uses the minumum amount of queries needed to fetch the inherited models.
Disabling polymorphic behavior when needed.

While django-polymorphic makes subclassed models easy to use in Django, we still encourage to use them with caution. Each subclassed model will require Django to perform an INNER JOIN to fetch the model fields from the database. While taking this in mind, there are valid reasons for using subclassed models. That's what this library is designed for!

The current release of django-polymorphic supports Django 2.1, 2.2, 3.0, 3.1 and Python 3.5+ is supported. For older Django versions, install django-polymorphic==1.3.

For more information, see the documentation at Read the Docs.

Installation

Install using pip...

$ pip install django-polymorphic

License

Django-polymorphic uses the same license as Django (BSD-like).

Comments

No longer actively maintained?

Hi,

This library has been a great help on projects I've worked on, but it looks like the last true merged PR was a very minor admin fix in July, 2019, and issues and open PRs are piling up.

@vdboor are you planning on returning to active maintenance or would you be fine with passing off active maintenance of this library to someone else? Are there any actively maintained forks that could be leveraged?

This library is fantastic, but could use some loving.

Thanks

opened by dmastylo 27
Django19

This adds support for django 1.9, as well as dropping some other deprecated features.

Closes #166. Closes #164.

This changeset has backwards-incompatible changes. I suggest a pushing the major in next release after merging this.

Noticably import path for models, managers, etc has been moved (both to stay in line with django standards, and to avoid crashes).

opened by WhyNotHugo 26

Prefetch_related has strange behavior

Hi,

I use django-polymorphic for a personal, non-critical project, in which items (Storable) can be stored in workspaces (Workspace).

This project relies heavily on concrete inheritance, and my models all inherit from a base model (using Polymorphic).

Everything works fine but, as always with concrete inheritance, database load is quite high, especially when I want to display a list of Storable, along with their workspaces. In order to reduce the number of queries, I'm trying to use prefetch_related('workspaces') on Storable queryset. However, while it reduces indeed the number of queries, there is a bug when come the time to display items.

I've set up a test project with the following code.

Models

from django.db import models
from polymorphic import PolymorphicModel


class Base(PolymorphicModel):
    name = models.CharField('name', max_length=255, default=None, blank=True, null=True)


class Storable(Base):
    pass


class Workspace(Base):
    items = models.ManyToManyField(Storable, related_name="workspaces", null=True, blank=True, default = None)

View

from django.views.generic.list import ListView
from test_prefetch_related.models import Storable


class TestPrefetchRelated(ListView):
    model = Storable
    template_name = "test_prefetch_related/test_prefetch_related.html"
    paginate_by = 10

    def get_queryset(self):
        return self.model.objects.all().prefetch_related('workspaces')

Template

<!DOCTYPE html>
<html>
<head>
    <title></title>
</head>
<body>
    {% for item in object_list %}
        <h1>{{ item.name }}</h1>
        <ul>
            {% for w in item.workspaces.all %}
                <li>{{ w.id }} - {{ w.name }}</li>
            {% empty %}
                None
            {% endfor %}
        </ul>
    {% endfor %}
</body>
</html>

Database population

from test_prefetch_related.models import Storable, Workspace

# here is the code I used to populate the database with some data

# create workspaces
w1 = Workspace(name="w1")
w1.save()

w2 = Workspace(name="w2")
w2.save()

w3 = Workspace(name="w3")
w3.save()

# create storables
s1 = Storable(name="s1")
s1.save()
s1.workspaces.add(w1)
s1.save()

s2 = Storable(name="s2")
s2.save()
s2.workspaces.add(w2)
s2.save()

s3 = Storable(name="s3")
s3.save()
s3.workspaces.add(w1, w2, w3)  # this storable get all workpaces
s3.save()

Html output

The bug

It's hard to explain (excuse my english ;), but as you can see in output, workspaces are not displayed properly, next to each Storable. Intead, they are grouped on the first item in the queryset with which they are linked.

The expected output would be :

I'm almost sure it is related to polymorphic because the following code using standard django inheritance works as expected.

In polymorphic docs, I did not found any mention of prefetch_related(). Is this behaviour one of the caveats of using concrete inheritance ? Is there anything else I could do to achieve the same result ?

You can download the code I used, if you want to check by yourself : http://seafile.eliotberriot.com/d/aca8f2399a/ (click on ZIP button in top right corner).

Thank you for your help and for this project, apart from that, I'm really happy with it !

help wanted

opened by agateblue 24

Add django 2.0 support
This fixes a few minor issues to add support for django 2.0(b1).

I have:

Removed django 1.10 from the test suite in the process, ~but have not yet dug out compat fixes that were added for it~.

Removed three models from the tests that were never referenced in the tests.

Not yet tested this manually.

There is one remaining unaddressed RemovedInDjango20Warning, but I've had some frustration with it. See https://code.djangoproject.com/ticket/28750

TODO:

[x] Remove django 1.10 compat fixes.

[x] Chase up related django ticket.
opened by meshy 20

Compatibility with Django 1.10: Attribute error

When running django-polymorphic on upcoming Django 1.10, following error is raised:

Traceback (most recent call last):
  File "runtests.py", line 83, in <module>
    runtests()
  File "runtests.py", line 80, in runtests
    execute_from_command_line(argv)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/apps/config.py", line 206, in import_models
    self.models_module = import_module(models_module_name)
  File "/opt/python/2.7.9/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/travis/build/PetrDlouhy/django-polymorphic/polymorphic/models.py", line 31, in <module>
    class PolymorphicModel(six.with_metaclass(PolymorphicModelBase, models.Model)):
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/utils/six.py", line 808, in __new__
    return meta(name, bases, d)
  File "/home/travis/build/PetrDlouhy/django-polymorphic/polymorphic/base.py", line 90, in __new__
    new_class._default_manager = user_manager._copy_to_model(new_class)
AttributeError: 'PolymorphicManager' object has no attribute '_copy_to_model'

opened by PetrDlouhy 20

Proxied models, delete() and PEP8

I worked on these things:

Better support for delete() in the QuerySets (and a way to "globally" disable polymorphism)

Added proxied polymorphic models.

PEP 8 cleaning

opened by Kronuz 18

Fix accessor replacement consistency on Django 1.11

At this point, we don't know why this line of code is executed, but we do know it's not consistently executed between Django 1.10 and Django 1.11 due to the addition of ForwardOneToOneDescriptor, a subclass of ForwardManyToOneDescriptor.

Refs. 6628145af7e894f6977e3a14a713ac14449f3a4e Refs. dj-stripe/dj-stripe#524 Refs. django/django@38575b007a722d6af510ea46d46393a4cda9ca29

opened by jleclanche 17

Django compatibility cleanups and fixes

This PR does a few things. In the first two commits:

We no longer test on versions of Django and Python which are no longer supported.
Remove all pre-django 1.8 hacks.

Then, in a separate commit, we remove support for Django 1.8 as well. I kept this separate because I may understand the argument to keep Django 1.8 as it is still supported upstream... however, Django 1.11 is the new LTS and, given the complexity of django-polymorphic and the amount of hacks dropping 1.8 removes, I would very highly recommend dropping 1.8 support as well, and recommend django-polymorphic==1.2 to Django 1.8 users.

As a followup to this, I am also improving compatibility with Django 2.0, to the point that tests now pass on django-dev. I cannot vouch for their reliability, however, especially given that I still see the following warning on 1.11 with -Wall:

Using Django version 1.11.1 from /home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django
/home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django/db/models/base.py:363: RemovedInDjango20Warning: Managers from concrete parents will soon qualify as default managers if they appear before any other managers in the MRO. As a result, 'base_objects' declared on 'polymorphic.PolymorphicModel' will no longer be the default manager for 'polymorphic.MROBase2' in favor of 'objects' declared on 'polymorphic.MROBase1'. You can redeclare 'base_objects' on 'MROBase2' to keep things the way they are or you can switch to the new behavior right away by setting `Meta.manager_inheritance_from_future` to `True`.
  if not opts.managers or cls._requires_legacy_default_manager():
/home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django/db/models/base.py:363: RemovedInDjango20Warning: Managers from concrete parents will soon qualify as default managers. As a result, the 'objects' manager won't be created (or recreated) automatically anymore on 'polymorphic.MgrInheritC' and 'mgrB' declared on 'polymorphic.MgrInheritB' will be promoted to default manager. You can declare explicitly `objects = models.Manager()` on 'MgrInheritC' to keep things the way they are or you can switch to the new behavior right away by setting `Meta.manager_inheritance_from_future` to `True`.
  if not opts.managers or cls._requires_legacy_default_manager():

Closes #287

opened by jleclanche 17

Test against django 1.11(b1) / python 3.6

This builds django-polymorphic against python 3.6 and django 1.11b1.

The new version of python works fine, but django 1.11 fails.

~~The test suite now runs checks, and it seems to be failing on one of them:~~ ~~polymorphic.MRODerived: (models.E005) The field 'id' from parent model 'polymorphic.mrobase3' clashes with the field 'id' from parent model 'polymorphic.mrobase1'.~~ ~~Once that's addressed, I suspect we're going to get a little closer to the cause of #267.~~

Ok, fixed that, now we're seeing #267.

opened by meshy 15
Fix prefetch_related behavior

I stumbled upon https://github.com/django-polymorphic/django-polymorphic/issues/68 which causes some errors for us. As mentioned in the issue, the problem is cause by reusing the instances stored in base_result_objects_by_id. This PR tries to work around it by maintaining the lists differently which allows to make shallow copies of the real instances (see line 417). Feel free to test!

opened by mgrrx 13
Please provide docs how to upgrade an existing model

If we change the parent class from models.Model to PolymorphicModel a new column gets added.

Please give us some hints how to fill the rows which already exist in the database.

Thank you.

opened by guettli 13

Child can't Many To Many Parent

class Parent(PolymorphicModel):
    ...

class Child(Parent):
    spam = models.ManyToManyField(Parent)

$ ./manage.py makemigrations
SystemCheckError: System check identified some issues:

ERRORS:
myapp.Child.spam: (fields.E305) Reverse query name for 'myapp.Child.spam' clashes with reverse query name for 'myapp.Child.parent_ptr'.
	HINT: Add or change a related_name argument to the definition for 'myapp.Child.spam' or 'myapp.Child.parent_ptr'.

opened by AstraLuma 0

Using non-polymorphic mixins breaks _base_manager

Here are three polymorphic models, each with different mixins:

class ModelMixin(models.Model):
    class Meta:
        abstract = True

    created_at = models.DateTimeField(auto_now_add=True)
    modified_at = models.DateTimeField(auto_now=True)

class PolymorphicMixin(PolymorphicModel):
    class Meta:
        abstract = True

    created_at = models.DateTimeField(auto_now_add=True)
    modified_at = models.DateTimeField(auto_now=True)


class Foo(PolymorphicModel):
    pass

class Bar(PolymorphicMixin, PolymorphicModel):
    pass

class Baz(ModelMixin, PolymorphicModel):
    pass

Both Foo and Bar have a base_manager of PolymorphicManager:

print(type(Foo._base_manager))  # <class 'polymorphic.managers.PolymorphicManager'>
print(type(Bar._base_manager))  # <class 'polymorphic.managers.PolymorphicManager'>

However, for Baz, its base_manager is not PolymorphicManager:

print(type(Baz._base_manager))  # <class 'django.db.models.manager.Manager'>

The reason I would use a non-polymorphic mixin is because I want to use it on multiple models, polymorphic and non-polymorphic alike. The only solution I've found is to duplicate my model mixin code, just swapping out models.Model with PolymorphicModel. I'd rather have only one mixin, do you know if this is possible?

opened by CelestialGuru 0

child models show in index regardless of show_in_index settings

from django.contrib import admin
from polymorphic.admin import PolymorphicParentModelAdmin, PolymorphicChildModelAdmin, PolymorphicChildModelFilter
from .models import Device, DeviceSubscribedEmail, Hub, Plug, PlugMini


class DeviceChildAdmin(PolymorphicChildModelAdmin):
    show_in_index = False

    def has_add_permission(self, request):
        return False

    def has_delete_permission(self, request, obj=None):
        return False


@admin.register(Plug)
class PlugAdmin(DeviceChildAdmin):
    base_model = Plug


class EmailSubscribersInline(admin.StackedInline):
    model = DeviceSubscribedEmail
    extra = 1


@admin.register(PlugMini)
class PlugMiniAdmin(DeviceChildAdmin):
    base_model = PlugMini
    fieldsets = [
        ("info", {"fields": ["id", "name", "country", "hub_device", "is_powered_on"]}),
        (
            "usage",
            {"fields": ["electricity_of_day", "voltage", "weight", "electric_current"]},
        ),
        ("thresholds", {"fields": ["weight_threshold"]}),
    ]
    inlines = [EmailSubscribersInline]


@admin.register(Hub)
class HubAdmin(DeviceChildAdmin):
    pass


@admin.register(Device)
class DeviceAdmin(PolymorphicParentModelAdmin):
    base_model = Device
    child_models = [PlugMini, Plug, Hub]
    list_filter = [PolymorphicChildModelFilter]

    def has_add_permission(self, request):
        return False

All models inherit from Device model which is a PolymorphicModel and still, all models show up in the admin page.

opened by LeOndaz 1

Support select_related and prefetch_related for inherited models and custom queryset for child models

add select_polymorphic_related, prefetch_polymorphic_related and custom_queryset methods for select and prefetch on inherited models

Triggers Django errors when that field doesn't exist in a particular subclass

Example works exactly same as django select_related and prefetch_related

queryset = Project.objects.select_polymorphic_related(
    ArtProject, 'artist', 'canvas__painter'
).select_polymorphic_related(
    ResearchProject, 'supervisor',
)


queryset = Project.objects.prefetch_polymorphic_related(
    ArtProject, 'artist', Prefetch('canvas', queryset=Project.objects.annotate(size=F('width') * F('height')))
).prefetch_polymorphic_related(
    ResearchProject, 'authors',
)


queryset = Project.objects.custom_queryset(ArtProject, ArtProject._base_objects.annotate(cost=F('cost') + F('canvas_cost'))))

opened by k4anubhav 5

Possible issue when using django-polymorphic along with typeddjango

Hi, this ticket is echoes https://github.com/typeddjango/django-stubs/issues/1158 which I opened earlier today on the typeddjango repo.

In short and as explained there, mypy (I'm using the django stubs) started to complain when I started to use django-polymorphic. Mypy complained that it couldn't resolve the manager type in classes inheriting from the polymorphic model.

Do I need to do something specific to get this to work well with mypy? I tried to change base_manager_name in the child classes to try to use a non_polymorphic manager but that didn't solve my issue.

I'm raising this ticket in this repo in case this issue comes from here and not from the typeddjango project. I might also have done something wrong. I'd welcome feedback if any.

Thanks in advance!

opened by AntoineRondelet 0

Improved Django model inheritance with automatic downcasting

Related tags

Overview

Polymorphic Models for Django

Features

Installation

License

Comments

Models

View

Template

Database population

Html output

The bug

Owner

Automatic caching and invalidation for Django models through the ORM.

An automatic django's update checker and MS teams notifier

Automatic class scheduler for Texas A&M written with Python+Django and React+Typescript

django-reversion is an extension to the Django web framework that provides version control for model instances.

Meta package to combine turbo-django and stimulus-django

Django-environ allows you to utilize 12factor inspired environment variables to configure your Django application.

Rosetta is a Django application that eases the translation process of your Django projects

Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly.

Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

django-quill-editor makes Quill.js easy to use on Django Forms and admin sites

A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, celery and redis.

A handy tool for generating Django-based backend projects without coding. On the other hand, it is a code generator of the Django framework.

A beginner django project and also my first Django project which involves shortening of a longer URL into a short one using a unique id.

Dockerizing Django with Postgres, Gunicorn, Nginx and Certbot. A fully Django starter project.

pytest-django allows you to test your Django project/applications with the pytest testing tool.

APIs for a Chat app. Written with Django Rest framework and Django channels.

django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project. Inspired in the dashboard framework Dashing

Django-MySQL extends Django's built-in MySQL and MariaDB support their specific features not available on other databases.

Django-Audiofield is a simple app that allows Audio files upload, management and conversion to different audio format (mp3, wav & ogg), which also makes it easy to play audio files into your Django application.