Improved Django model inheritance with automatic downcasting

Overview
https://travis-ci.org/django-polymorphic/django-polymorphic.svg?branch=master https://readthedocs.org/projects/django-polymorphic/badge/?version=stable

Polymorphic Models for Django

Django-polymorphic simplifies using inherited models in Django projects. When a query is made at the base model, the inherited model classes are returned.

When we store models that inherit from a Project model...

>>> Project.objects.create(topic="Department Party")
>>> ArtProject.objects.create(topic="Painting with Tim", artist="T. Turner")
>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")

...and want to retrieve all our projects, the subclassed models are returned!

>>> Project.objects.all()
[ <Project:         id 1, topic "Department Party">,
  <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner">,
  <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]

Using vanilla Django, we get the base class objects, which is rarely what we wanted:

>>> Project.objects.all()
[ <Project: id 1, topic "Department Party">,
  <Project: id 2, topic "Painting with Tim">,
  <Project: id 3, topic "Swallow Aerodynamics"> ]

This also works when the polymorphic model is accessed via ForeignKeys, ManyToManyFields or OneToOneFields.

Features

  • Full admin integration.
  • ORM integration:
    • support for ForeignKey, ManyToManyField, OneToOneField descriptors.
    • Filtering/ordering of inherited models (ArtProject___artist).
    • Filtering model types: instance_of(...) and not_instance_of(...)
    • Combining querysets of different models (qs3 = qs1 | qs2)
    • Support for custom user-defined managers.
  • Uses the minumum amount of queries needed to fetch the inherited models.
  • Disabling polymorphic behavior when needed.

While django-polymorphic makes subclassed models easy to use in Django, we still encourage to use them with caution. Each subclassed model will require Django to perform an INNER JOIN to fetch the model fields from the database. While taking this in mind, there are valid reasons for using subclassed models. That's what this library is designed for!

The current release of django-polymorphic supports Django 2.1, 2.2, 3.0, 3.1 and Python 3.5+ is supported. For older Django versions, install django-polymorphic==1.3.

For more information, see the documentation at Read the Docs.

Installation

Install using pip...

$ pip install django-polymorphic

License

Django-polymorphic uses the same license as Django (BSD-like).

Comments
  • No longer actively maintained?

    No longer actively maintained?

    Hi,

    This library has been a great help on projects I've worked on, but it looks like the last true merged PR was a very minor admin fix in July, 2019, and issues and open PRs are piling up.

    @vdboor are you planning on returning to active maintenance or would you be fine with passing off active maintenance of this library to someone else? Are there any actively maintained forks that could be leveraged?

    This library is fantastic, but could use some loving.

    Thanks

    opened by dmastylo 27
  • Django19

    Django19

    This adds support for django 1.9, as well as dropping some other deprecated features.

    Closes #166. Closes #164.

    This changeset has backwards-incompatible changes. I suggest a pushing the major in next release after merging this.

    Noticably import path for models, managers, etc has been moved (both to stay in line with django standards, and to avoid crashes).

    opened by WhyNotHugo 26
  • Prefetch_related has strange behavior

    Prefetch_related has strange behavior

    Hi,

    I use django-polymorphic for a personal, non-critical project, in which items (Storable) can be stored in workspaces (Workspace).

    This project relies heavily on concrete inheritance, and my models all inherit from a base model (using Polymorphic).

    Everything works fine but, as always with concrete inheritance, database load is quite high, especially when I want to display a list of Storable, along with their workspaces. In order to reduce the number of queries, I'm trying to use prefetch_related('workspaces') on Storable queryset. However, while it reduces indeed the number of queries, there is a bug when come the time to display items.

    I've set up a test project with the following code.

    Models

    from django.db import models
    from polymorphic import PolymorphicModel
    
    
    class Base(PolymorphicModel):
        name = models.CharField('name', max_length=255, default=None, blank=True, null=True)
    
    
    class Storable(Base):
        pass
    
    
    class Workspace(Base):
        items = models.ManyToManyField(Storable, related_name="workspaces", null=True, blank=True, default = None)
    

    View

    from django.views.generic.list import ListView
    from test_prefetch_related.models import Storable
    
    
    class TestPrefetchRelated(ListView):
        model = Storable
        template_name = "test_prefetch_related/test_prefetch_related.html"
        paginate_by = 10
    
        def get_queryset(self):
            return self.model.objects.all().prefetch_related('workspaces')
    

    Template

    <!DOCTYPE html>
    <html>
    <head>
        <title></title>
    </head>
    <body>
        {% for item in object_list %}
            <h1>{{ item.name }}</h1>
            <ul>
                {% for w in item.workspaces.all %}
                    <li>{{ w.id }} - {{ w.name }}</li>
                {% empty %}
                    None
                {% endfor %}
            </ul>
        {% endfor %}
    </body>
    </html>
    

    Database population

    from test_prefetch_related.models import Storable, Workspace
    
    # here is the code I used to populate the database with some data
    
    # create workspaces
    w1 = Workspace(name="w1")
    w1.save()
    
    w2 = Workspace(name="w2")
    w2.save()
    
    w3 = Workspace(name="w3")
    w3.save()
    
    # create storables
    s1 = Storable(name="s1")
    s1.save()
    s1.workspaces.add(w1)
    s1.save()
    
    s2 = Storable(name="s2")
    s2.save()
    s2.workspaces.add(w2)
    s2.save()
    
    s3 = Storable(name="s3")
    s3.save()
    s3.workspaces.add(w1, w2, w3)  # this storable get all workpaces
    s3.save()
    

    Html output

    s1
    
        1 - w1
        1 - w1
    
    s2
    
        2 - w2
        2 - w2
    
    s3
    
        3 - w3
    

    The bug

    It's hard to explain (excuse my english ;), but as you can see in output, workspaces are not displayed properly, next to each Storable. Intead, they are grouped on the first item in the queryset with which they are linked.

    The expected output would be :

    s1
    
        1 - w1
    
    s2
    
        2 - w2
    
    s3     
    
        1 - w1
        2 - w2
        3 - w3
    

    I'm almost sure it is related to polymorphic because the following code using standard django inheritance works as expected.

    In polymorphic docs, I did not found any mention of prefetch_related(). Is this behaviour one of the caveats of using concrete inheritance ? Is there anything else I could do to achieve the same result ?

    You can download the code I used, if you want to check by yourself : http://seafile.eliotberriot.com/d/aca8f2399a/ (click on ZIP button in top right corner).

    Thank you for your help and for this project, apart from that, I'm really happy with it !

    help wanted 
    opened by agateblue 24
  • Add django 2.0 support

    Add django 2.0 support

    This fixes a few minor issues to add support for django 2.0(b1).

    I have:

    • Removed django 1.10 from the test suite in the process, ~but have not yet dug out compat fixes that were added for it~.
    • Removed three models from the tests that were never referenced in the tests.
    • Not yet tested this manually.

    There is one remaining unaddressed RemovedInDjango20Warning, but I've had some frustration with it. See https://code.djangoproject.com/ticket/28750

    TODO:

    • [x] Remove django 1.10 compat fixes.
    • [x] Chase up related django ticket.
    opened by meshy 20
  • Compatibility with Django 1.10: Attribute error

    Compatibility with Django 1.10: Attribute error

    When running django-polymorphic on upcoming Django 1.10, following error is raised:

    Traceback (most recent call last):
      File "runtests.py", line 83, in <module>
        runtests()
      File "runtests.py", line 80, in runtests
        execute_from_command_line(argv)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
        utility.execute()
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 341, in execute
        django.setup()
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
        apps.populate(settings.INSTALLED_APPS)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
        app_config.import_models(all_models)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/apps/config.py", line 206, in import_models
        self.models_module = import_module(models_module_name)
      File "/opt/python/2.7.9/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/home/travis/build/PetrDlouhy/django-polymorphic/polymorphic/models.py", line 31, in <module>
        class PolymorphicModel(six.with_metaclass(PolymorphicModelBase, models.Model)):
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/utils/six.py", line 808, in __new__
        return meta(name, bases, d)
      File "/home/travis/build/PetrDlouhy/django-polymorphic/polymorphic/base.py", line 90, in __new__
        new_class._default_manager = user_manager._copy_to_model(new_class)
    AttributeError: 'PolymorphicManager' object has no attribute '_copy_to_model'
    
    opened by PetrDlouhy 20
  • Proxied models, delete() and PEP8

    Proxied models, delete() and PEP8

    I worked on these things:

    Better support for delete() in the QuerySets (and a way to "globally" disable polymorphism)
    
    Added proxied polymorphic models.
    
    PEP 8 cleaning
    
    opened by Kronuz 18
  • Fix accessor replacement consistency on Django 1.11

    Fix accessor replacement consistency on Django 1.11

    At this point, we don't know why this line of code is executed, but we do know it's not consistently executed between Django 1.10 and Django 1.11 due to the addition of ForwardOneToOneDescriptor, a subclass of ForwardManyToOneDescriptor.

    Refs. 6628145af7e894f6977e3a14a713ac14449f3a4e Refs. dj-stripe/dj-stripe#524 Refs. django/django@38575b007a722d6af510ea46d46393a4cda9ca29

    opened by jleclanche 17
  • Django compatibility cleanups and fixes

    Django compatibility cleanups and fixes

    This PR does a few things. In the first two commits:

    • We no longer test on versions of Django and Python which are no longer supported.
    • Remove all pre-django 1.8 hacks.

    Then, in a separate commit, we remove support for Django 1.8 as well. I kept this separate because I may understand the argument to keep Django 1.8 as it is still supported upstream... however, Django 1.11 is the new LTS and, given the complexity of django-polymorphic and the amount of hacks dropping 1.8 removes, I would very highly recommend dropping 1.8 support as well, and recommend django-polymorphic==1.2 to Django 1.8 users.

    As a followup to this, I am also improving compatibility with Django 2.0, to the point that tests now pass on django-dev. I cannot vouch for their reliability, however, especially given that I still see the following warning on 1.11 with -Wall:

    Using Django version 1.11.1 from /home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django
    /home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django/db/models/base.py:363: RemovedInDjango20Warning: Managers from concrete parents will soon qualify as default managers if they appear before any other managers in the MRO. As a result, 'base_objects' declared on 'polymorphic.PolymorphicModel' will no longer be the default manager for 'polymorphic.MROBase2' in favor of 'objects' declared on 'polymorphic.MROBase1'. You can redeclare 'base_objects' on 'MROBase2' to keep things the way they are or you can switch to the new behavior right away by setting `Meta.manager_inheritance_from_future` to `True`.
      if not opts.managers or cls._requires_legacy_default_manager():
    /home/adys/tmp/django-polymorphic/.tox/py36-django111/lib/python3.6/site-packages/django/db/models/base.py:363: RemovedInDjango20Warning: Managers from concrete parents will soon qualify as default managers. As a result, the 'objects' manager won't be created (or recreated) automatically anymore on 'polymorphic.MgrInheritC' and 'mgrB' declared on 'polymorphic.MgrInheritB' will be promoted to default manager. You can declare explicitly `objects = models.Manager()` on 'MgrInheritC' to keep things the way they are or you can switch to the new behavior right away by setting `Meta.manager_inheritance_from_future` to `True`.
      if not opts.managers or cls._requires_legacy_default_manager():
    

    Closes #287

    opened by jleclanche 17
  • Test against django 1.11(b1) / python 3.6

    Test against django 1.11(b1) / python 3.6

    This builds django-polymorphic against python 3.6 and django 1.11b1.

    The new version of python works fine, but django 1.11 fails.

    ~~The test suite now runs checks, and it seems to be failing on one of them:~~ ~~polymorphic.MRODerived: (models.E005) The field 'id' from parent model 'polymorphic.mrobase3' clashes with the field 'id' from parent model 'polymorphic.mrobase1'.~~ ~~Once that's addressed, I suspect we're going to get a little closer to the cause of #267.~~

    Ok, fixed that, now we're seeing #267.

    opened by meshy 15
  • Fix prefetch_related behavior

    Fix prefetch_related behavior

    I stumbled upon https://github.com/django-polymorphic/django-polymorphic/issues/68 which causes some errors for us. As mentioned in the issue, the problem is cause by reusing the instances stored in base_result_objects_by_id. This PR tries to work around it by maintaining the lists differently which allows to make shallow copies of the real instances (see line 417). Feel free to test!

    opened by mgrrx 13
  • Please provide docs how to upgrade an existing model

    Please provide docs how to upgrade an existing model

    If we change the parent class from models.Model to PolymorphicModel a new column gets added.

    Please give us some hints how to fill the rows which already exist in the database.

    Thank you.

    opened by guettli 13
  • Child can't Many To Many Parent

    Child can't Many To Many Parent

    class Parent(PolymorphicModel):
        ...
    
    class Child(Parent):
        spam = models.ManyToManyField(Parent)
    
    $ ./manage.py makemigrations
    SystemCheckError: System check identified some issues:
    
    ERRORS:
    myapp.Child.spam: (fields.E305) Reverse query name for 'myapp.Child.spam' clashes with reverse query name for 'myapp.Child.parent_ptr'.
    	HINT: Add or change a related_name argument to the definition for 'myapp.Child.spam' or 'myapp.Child.parent_ptr'.
    
    opened by AstraLuma 0
  • Using non-polymorphic mixins breaks _base_manager

    Using non-polymorphic mixins breaks _base_manager

    Here are three polymorphic models, each with different mixins:

    class ModelMixin(models.Model):
        class Meta:
            abstract = True
    
        created_at = models.DateTimeField(auto_now_add=True)
        modified_at = models.DateTimeField(auto_now=True)
    
    class PolymorphicMixin(PolymorphicModel):
        class Meta:
            abstract = True
    
        created_at = models.DateTimeField(auto_now_add=True)
        modified_at = models.DateTimeField(auto_now=True)
    
    
    class Foo(PolymorphicModel):
        pass
    
    class Bar(PolymorphicMixin, PolymorphicModel):
        pass
    
    class Baz(ModelMixin, PolymorphicModel):
        pass
    

    Both Foo and Bar have a base_manager of PolymorphicManager:

    print(type(Foo._base_manager))  # <class 'polymorphic.managers.PolymorphicManager'>
    print(type(Bar._base_manager))  # <class 'polymorphic.managers.PolymorphicManager'>
    

    However, for Baz, its base_manager is not PolymorphicManager:

    print(type(Baz._base_manager))  # <class 'django.db.models.manager.Manager'>
    

    The reason I would use a non-polymorphic mixin is because I want to use it on multiple models, polymorphic and non-polymorphic alike. The only solution I've found is to duplicate my model mixin code, just swapping out models.Model with PolymorphicModel. I'd rather have only one mixin, do you know if this is possible?

    opened by CelestialGuru 0
  • child models show in index regardless of show_in_index settings

    child models show in index regardless of show_in_index settings

    from django.contrib import admin
    from polymorphic.admin import PolymorphicParentModelAdmin, PolymorphicChildModelAdmin, PolymorphicChildModelFilter
    from .models import Device, DeviceSubscribedEmail, Hub, Plug, PlugMini
    
    
    class DeviceChildAdmin(PolymorphicChildModelAdmin):
        show_in_index = False
    
        def has_add_permission(self, request):
            return False
    
        def has_delete_permission(self, request, obj=None):
            return False
    
    
    @admin.register(Plug)
    class PlugAdmin(DeviceChildAdmin):
        base_model = Plug
    
    
    class EmailSubscribersInline(admin.StackedInline):
        model = DeviceSubscribedEmail
        extra = 1
    
    
    @admin.register(PlugMini)
    class PlugMiniAdmin(DeviceChildAdmin):
        base_model = PlugMini
        fieldsets = [
            ("info", {"fields": ["id", "name", "country", "hub_device", "is_powered_on"]}),
            (
                "usage",
                {"fields": ["electricity_of_day", "voltage", "weight", "electric_current"]},
            ),
            ("thresholds", {"fields": ["weight_threshold"]}),
        ]
        inlines = [EmailSubscribersInline]
    
    
    @admin.register(Hub)
    class HubAdmin(DeviceChildAdmin):
        pass
    
    
    @admin.register(Device)
    class DeviceAdmin(PolymorphicParentModelAdmin):
        base_model = Device
        child_models = [PlugMini, Plug, Hub]
        list_filter = [PolymorphicChildModelFilter]
    
        def has_add_permission(self, request):
            return False
    
    

    All models inherit from Device model which is a PolymorphicModel and still, all models show up in the admin page.

    opened by LeOndaz 1
  • Support select_related and prefetch_related for inherited models and custom queryset for child models

    Support select_related and prefetch_related for inherited models and custom queryset for child models

    add select_polymorphic_related, prefetch_polymorphic_related and custom_queryset methods for select and prefetch on inherited models

    Triggers Django errors when that field doesn't exist in a particular subclass

    Example works exactly same as django select_related and prefetch_related

    queryset = Project.objects.select_polymorphic_related(
        ArtProject, 'artist', 'canvas__painter'
    ).select_polymorphic_related(
        ResearchProject, 'supervisor',
    )
    
    
    queryset = Project.objects.prefetch_polymorphic_related(
        ArtProject, 'artist', Prefetch('canvas', queryset=Project.objects.annotate(size=F('width') * F('height')))
    ).prefetch_polymorphic_related(
        ResearchProject, 'authors',
    )
    
    
    queryset = Project.objects.custom_queryset(ArtProject, ArtProject._base_objects.annotate(cost=F('cost') + F('canvas_cost'))))
    
    opened by k4anubhav 5
  • Possible issue when using django-polymorphic along with typeddjango

    Possible issue when using django-polymorphic along with typeddjango

    Hi, this ticket is echoes https://github.com/typeddjango/django-stubs/issues/1158 which I opened earlier today on the typeddjango repo.

    In short and as explained there, mypy (I'm using the django stubs) started to complain when I started to use django-polymorphic. Mypy complained that it couldn't resolve the manager type in classes inheriting from the polymorphic model.

    Do I need to do something specific to get this to work well with mypy? I tried to change base_manager_name in the child classes to try to use a non_polymorphic manager but that didn't solve my issue.

    I'm raising this ticket in this repo in case this issue comes from here and not from the typeddjango project. I might also have done something wrong. I'd welcome feedback if any.

    Thanks in advance!

    opened by AntoineRondelet 0
Owner
Improved Django Model Inheritance
null
Automatic caching and invalidation for Django models through the ORM.

Cache Machine Cache Machine provides automatic caching and invalidation for Django models through the ORM. For full docs, see https://cache-machine.re

null 846 Nov 26, 2022
An automatic django's update checker and MS teams notifier

Django Update Checker This is small script for checking any new updates/bugfixes/security fixes released in django News & Events and sending correspon

prinzpiuz 4 Sep 26, 2022
Automatic class scheduler for Texas A&M written with Python+Django and React+Typescript

Rev Registration Description Rev Registration is an automatic class scheduler for Texas A&M, aimed at easing the process of course registration by gen

Aggie Coding Club 21 Nov 15, 2022
django-reversion is an extension to the Django web framework that provides version control for model instances.

django-reversion django-reversion is an extension to the Django web framework that provides version control for model instances. Requirements Python 3

Dave Hall 2.8k Jan 2, 2023
Meta package to combine turbo-django and stimulus-django

Hotwire + Django This repository aims to help you integrate Hotwire with Django ?? Inspiration might be taken from @hotwired/hotwire-rails. We are sti

Hotwire for Django 31 Aug 9, 2022
Django-environ allows you to utilize 12factor inspired environment variables to configure your Django application.

Django-environ django-environ allows you to use Twelve-factor methodology to configure your Django application with environment variables. import envi

Daniele Faraglia 2.7k Jan 7, 2023
Rosetta is a Django application that eases the translation process of your Django projects

Rosetta Rosetta is a Django application that facilitates the translation process of your Django projects. Because it doesn't export any models, Rosett

Marco Bonetti 909 Dec 26, 2022
Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly.

Cookiecutter Django Powered by Cookiecutter, Cookiecutter Django is a framework for jumpstarting production-ready Django projects quickly. Documentati

Daniel Feldroy 10k Dec 31, 2022
Django project starter on steroids: quickly create a Django app AND generate source code for data models + REST/GraphQL APIs (the generated code is auto-linted and has 100% test coverage).

Create Django App ?? We're a Django project starter on steroids! One-line command to create a Django app with all the dependencies auto-installed AND

imagine.ai 68 Oct 19, 2022
django-quill-editor makes Quill.js easy to use on Django Forms and admin sites

django-quill-editor django-quill-editor makes Quill.js easy to use on Django Forms and admin sites No configuration required for static files! The ent

lhy 139 Dec 5, 2022
A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, celery and redis.

Django Channels Websocket Chatbot A Django chatbot that is capable of doing math and searching Chinese poet online. Developed with django, channels, c

Yunbo Shi 8 Oct 28, 2022
A handy tool for generating Django-based backend projects without coding. On the other hand, it is a code generator of the Django framework.

Django Sage Painless The django-sage-painless is a valuable package based on Django Web Framework & Django Rest Framework for high-level and rapid web

sageteam 51 Sep 15, 2022
A beginner django project and also my first Django project which involves shortening of a longer URL into a short one using a unique id.

Django-URL-Shortener A beginner django project and also my first Django project which involves shortening of a longer URL into a short one using a uni

Rohini Rao 3 Aug 8, 2021
Dockerizing Django with Postgres, Gunicorn, Nginx and Certbot. A fully Django starter project.

Dockerizing Django with Postgres, Gunicorn, Nginx and Certbot ?? Features A Django stater project with fully basic requirements for a production-ready

null 8 Jun 27, 2022
pytest-django allows you to test your Django project/applications with the pytest testing tool.

pytest-django allows you to test your Django project/applications with the pytest testing tool.

pytest-dev 1.1k Dec 14, 2022
APIs for a Chat app. Written with Django Rest framework and Django channels.

ChatAPI APIs for a Chat app. Written with Django Rest framework and Django channels. The documentation for the http end points can be found here This

Victor Aderibigbe 18 Sep 9, 2022
django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project. Inspired in the dashboard framework Dashing

django-dashing django-dashing is a customisable, modular dashboard application framework for Django to visualize interesting data about your project.

talPor Solutions 703 Dec 22, 2022
Django-MySQL extends Django's built-in MySQL and MariaDB support their specific features not available on other databases.

Django-MySQL The dolphin-pony - proof that cute + cute = double cute. Django-MySQL extends Django's built-in MySQL and MariaDB support their specific

Adam Johnson 504 Jan 4, 2023
Django-Audiofield is a simple app that allows Audio files upload, management and conversion to different audio format (mp3, wav & ogg), which also makes it easy to play audio files into your Django application.

Django-Audiofield Description: Django Audio Management Tools Maintainer: Areski Contributors: list of contributors Django-Audiofield is a simple app t

Areski Belaid 167 Nov 10, 2022