Django application and library for importing and exporting data with admin integration.

Overview

django-import-export

Build status on Travis-CI https://coveralls.io/repos/github/django-import-export/django-import-export/badge.svg?branch=master Current version on PyPi Documentation

PyPI - Python Version

PyPI - Django Version

django-import-export is a Django application and library for importing and exporting data with included admin integration.

Features:

  • support multiple formats (Excel, CSV, JSON, ... and everything else that tablib supports)
  • admin integration for importing
  • preview import changes
  • admin integration for exporting
  • export data respecting admin filters

docs/_static/images/django-import-export-change.png

Example app

To run the demo app:

cd tests
./manage.py makemigrations
./manage.py migrate
./manage.py createsuperuser
./manage.py loaddata category book
./manage.py runserver

Contribute

If you'd like to contribute, simply fork the repository, commit your changes to the develop branch (or branch off of it), and send a pull request. Make sure you add yourself to AUTHORS.

As most projects, we try to follow PEP8 as closely as possible. Please bear in mind that most pull requests will be rejected without proper unit testing.

Issues
  • Better surfacing of validation errors in UI / optional model instance validation

    Better surfacing of validation errors in UI / optional model instance validation

    Hi @bmihelac,

    Related issues:

    • #64
    • #176
    • #415
    • #601

    Here's where I've got to so far. I thought it would be good to get some feedback before getting stuck into writing new tests, but the current ones are passing, so at least nothing is broken :)

    When enabled, instance.full_clean() is called for each row, and any resulting validation errors surface themselves in the preview table like so:

    screenshot 2018-11-02 at 15 35 37

    The errors are hidden visually at first, then become visible when you hover over a row's first column, in such a way that it doesn't hide any of the values for the current row. It's pretty crude, but it's a starting point. Some examples:

    screenshot 2018-11-02 at 15 35 42 screenshot 2018-11-02 at 15 35 53

    Anyone wanting to give this a whirl for themselves should be able to install from git using:

    pip install git+https://github.com/ababic/[email protected]

    Any/all feedback welcome :)

    opened by ababic 43
  • when import csv file ,KeyError: u'id' occured

    when import csv file ,KeyError: u'id' occured

    i try to import some data form csv file, but errors occurred as follows:

    Errors Line number: 1 - u'id' Traceback (most recent call last): File "/home/zhijiasun/.virtualenvs/tutorial/local/lib/python2.7/site-packages/import_export/resources.py", line 317, in import_data instance, new = self.get_or_init_instance(instance_loader, row) File "/home/zhijiasun/.virtualenvs/tutorial/local/lib/python2.7/site-packages/import_export/resources.py", line 149, in get_or_init_instance instance = self.get_instance(instance_loader, row) KeyError: u'id'

    I just modify the data of the csv file which exported by the app and then try to import it. But error happed.

    stale 
    opened by zhijiasun 36
  • question:UnicodeDecodeError at /admin/blog/author/import/

    question:UnicodeDecodeError at /admin/blog/author/import/

    Sir,thanks a lot to develop this great repository. I have little problem to solve , as below the code:

    #in models.py
    class Author(models.Model):
        author = models.CharField('作者', max_length=50)
        title = models.CharField('标题', max_length=150)
        qualification = models.ForeignKey(Qualification)
        mark = models.ManyToManyField(Mark)
        blog = models.TextField('博客内容')
        time = models.DateField('写作日期')
    
        def __unicode__(self):
            return unicode(self.author)
    
        class Meta:
            ordering = ['time']
    
    #in admin.py
    class AuthorAdmin(ImportExportModelAdmin, admin.ModelAdmin):
        search_fields = ('author', 'title', 'mark', 'blog')
        list_display = ('author', 'title', 'time')
    

    When I export the data, 1 but the csv file appear to be the retortion, 2 when I modified the second row author data and import,

    3 it cause the error, 4

    how I it be smoothly modified and import?thanks. Allenlee

    opened by hebijiandai 31
  • CLI import (reprise)

    CLI import (reprise)

    This is a continuation of @int-ua's work in #725, addressing #332. I've resolved conflicts, tweaked the documentation and tests, but otherwise kept it identical.

    I need this feature too, so I'd like to get it merged sooner rather than later. However, there are other tweaks I'd like to make, including:

    • Better documentation, including its own page in the Reference
    • Format options:
      • arg to list available formats, calling into base_formats rather than being hardcoded
      • arg to specify format to use rather than guessing
    • Detect whether supplied class is a Resource or Model rather than needing to specify
    • raise_errors currently defaults to True if dry_run is False - is this the right thing to do?
    • Option to supply kwargs to the Resource constructor (which I need for my own project)
    • Hey, export would be nice too

    ... but let's start with this, and if I get time for those tweaks I'll make them in a separate PR.

    enhancement needs update 
    opened by yozlet 27
  • Added ExportViewMixin

    Added ExportViewMixin

    Created a mixin view to export without admin interface.

    I think it would be a really great thing if we have a generic class based views for this application.

    I tested this patch with a ListView & django_filters.views.FilterView. The 2nd is really useful to replace the AdminFilters

    needs update 
    opened by ZuluPro 23
  • Many to many fields and admin upload

    Many to many fields and admin upload

    Do many to many fields work via admin imports? I keep getting something like this: ValueError('"<Accession: >" needs to have a value for field "accession" before this many-to-many relationship can be used.',)

    It doesn't seem like it can if it's displaying the data to import before importing, as there's no way to get the primary key for the row since it doesn't have one yet?

    Thanks,

    Dean

    opened by lfarrell 23
  • CLI import_file command

    CLI import_file command

    #332

    enhancement needs update 
    opened by int-ua 19
  • 1.8 compatibility

    1.8 compatibility

      File "/Users/syabro/.envs/homeowners/lib/python2.7/site-packages/import_export/resources.py", line 16, in <module>
        from django.db.models.related import RelatedObject
    ImportError: No module named related
    

    because

    The django.db.models.related module has been removed and the Field.related attribute will be removed in Django 2.0.

    https://docs.djangoproject.com/en/dev/releases/1.8/#model-field-related

    bug 
    opened by syabro 19
  • import error, 'get() keywords must be strings'

    import error, 'get() keywords must be strings'

    I'm having trouble getting a form to accept data on import. Currently testing csv format file; get same errors with xls file.

    sample from import.csv:

    jid,project_code,collab_id,sample_code,sequencing_type
    JMQN3H,AAST,42567,ST,1
    JO2FT1,AAST,43267,ST,1
    

    The error coming out is get() keywords must be strings.

    django error log:

    Line number: 1 - get() keywords must be strings
    OrderedDict([('jid', 'JMQN3H'), ('project_code', 'AAST'), ('collab_id', '42567'), ('sample_code', 'ST'), ('sequencing_type', '1')])
    Traceback (most recent call last):
    File "/var/www/django/jid_generator/lib/python3.4/site-packages/import_export/resources.py", line 356, in import_data
    instance, new = self.get_or_init_instance(instance_loader, row)
    File "/var/www/django/jid_generator/lib/python3.4/site-packages/import_export/resources.py", line 186, in get_or_init_instance
    instance = self.get_instance(instance_loader, row)
    TypeError: get() keywords must be strings
    

    I tried putting the values in csv file inside quotes. I tried modifying the admin, form and the Resource widgets.

    admin.py:

    """ImportExport Resource"""
    class DetailResource(resources.ModelResource):
        jid = fields.Field(
            default=generate_Jid(prefix='J'),
            readonly=True,
            widget=widgets.CharWidget(),
            )
        project_code = fields.Field(column_name='project_code', attribute='project_code',
            widget=widgets.ForeignKeyWidget(ProjectCode, 'code'),
            )
        sample_type = fields.Field(
            widget=widgets.ForeignKeyWidget(SampleType, 'code'),
            )
        sequencing_type = fields.Field(
            widget=widgets.ForeignKeyWidget(SequencingType, 'code'),
            )
        collab_id = fields.Field(
            widget=widgets.CharWidget(),
            )
    
        class Meta:
            model = JIdDetail
            all_fields = ( 'jid', 'project_code', 'collab_id',
                    'sample_type', 'sequencing_type', )
            import_id_fields = ( 'jid', )
            fields = all_fields
            export_order = all_fields
    
    @admin.register(JIdDetail)
    # class JIdDetailAdmin(admin.ModelAdmin):
    class JIdDetailAdmin(ImportExportModelAdmin):
        resource_class = DetailResource
    
        # has_add_permission removes the individual 'add' admin action
        # def has_add_permission(self, request):
            # return False
    
        form = JIdDetailForm
        actions_on_top = False
        actions = None
        all_fields = ( 'jid', 'project_code', 'collab_id',
                'sample_type', 'sequencing_type', )
        fields = ( all_fields, )
        list_display = all_fields
        search_fields = all_fields
        readonly_fields = ( 'jid' ),
        list_filter = ( 'sample_type', 'sequencing_type', 'project_code' )
        ordering = ['creation_date', 'project_code', 'sequencing_type']
    

    forms.py:

    class JIdDetailForm(forms.ModelForm):
        class Meta:
            model = JIdDetail
            fields = ( 'jid', 'project_code', 'collab_id',
                    'sample_type', 'sequencing_type', )
            readonly_fields = ( 'jid' ),
    
            # widgets = {
                # 'jid': forms.TextInput(attrs={'size':'6'}),
                # 'collab_id': forms.TextInput(attrs={'size':'40'}),
                # }
    

    models.py

    class JIdDetail(models.Model):
        verbose_name = 'J Id Detail'
        jid = models.CharField('J ID',
                max_length=6, blank=False,
                help_text="A unique ID string for every sample.",
                unique=True,
                )
        project_code = models.ForeignKey(ProjectCode, to_field='code')
        collab_id = models.TextField('Collaborator ID', blank=False,
                help_text="Collaborator sample ID."
                )
        sample_type = models.ForeignKey(SampleType, to_field='code')
        sequencing_type = models.ForeignKey(SequencingType, to_field='code')
        creation_date = models.DateTimeField(auto_now_add=True)
    

    Any tips on how to troubleshoot this?

    Thanks! -Benjamin-

    opened by cometsong 17
  • Exporting Decimal Values

    Exporting Decimal Values

    When i export my data, instead of 0 i get 0E-8 How can i fix this ? image

    question 
    opened by gorkemye 1
  • Explore Django Rest Framework extension using ModelSerializers

    Explore Django Rest Framework extension using ModelSerializers

    I was thinking it'd be pretty feasible to use the DRF ModelSerializers to greatly simplify some complex tasks (related to issue #1375 ) like importing/exporting subsets of data with arbitrary depth.

    So we could add a higher level of Resource that uses ModelSerializers or Serializers to import an model instance with its entire hierarchy of relationships intact, e.g. SerializerResource or the like.

    It's entirely possible that Serializer could allow moving away from needing an underlying dataset engine, although I'm not sure if performance is there at scale or not. They seem pretty fast though.

    (It's entirely possible that expanding to support another library in DRF is too big a scope, just an idea I had that might be worth some cycles)

    enhancement question design-decision-needed 
    opened by pokken-magic 0
  • Create a ResourceField that allows exporting and importing related subsets of data

    Create a ResourceField that allows exporting and importing related subsets of data

    (This is fairly involved -- trying to get all the random ideas I had on this down before I forget them)

    A Field created with a reference to a Resource could allow exporting an entire related model, which would enable (potentially) exporting subsets of data with arbitrary depth (where a resourcefield could have other resourcefields) --

    So if you had a Book with Publishers and Authors, you could export that Book and all of the Publishers and Authors necessary for it. ResourceFields would need to be imported first, which likely would require significant UI changes -- such as an additional workflow step for each level of depth (if Publishers have Representatives, you'd have to load Representatives before Publishers, then Publishers and Authors before Books).

    The power here is that you could create a more consistent dataset with complex realistic models, without having to take multiple steps to synchronize all the dependencies -- you just export a Book, and everything needed for that Book comes with it.

    enhancement question 
    opened by pokken-magic 1
  • Export from a specific model view page in Django Admin

    Export from a specific model view page in Django Admin

    Does anyone have any ideas on the best way to add an export button on a model instance's page in Django Admin? The thought being that if you browse to a resource, it would be fairly natural workflow-wise to want to be able to click Export on that page and export that model instance.

    I imagine that it is fairly easy to custom code this but it might be a cool feature as a mixin for the model admin, or as an additional feature for the ExportActionMixin.

    enhancement question 
    opened by pokken-magic 0
  • Allow default format selections on import, export, and action screens

    Allow default format selections on import, export, and action screens

    It would be useful to allow specifying a default format on the import and export screens as well as the actions. On some of the screens it will auto-select if you only have one option, but the Actions do not work that way. If you're always using a specific format (or usually even) it gets to be a lot of clicking going back to Yaml or Json or what have you from the menu.

    enhancement question good first time issue 
    opened by pokken-magic 0
  • Add Natural Key support to ForeignKeyWidget

    Add Natural Key support to ForeignKeyWidget

    Problem

    Foreign key widgets must be manually configured with a list of fields if using natural keys, vs. using the built in Django natural key functionality. Having this tight coupling between what is a natural key and what is specified in the resource requires changes in two places if a natural key function is changed, which seems undesirable.

    Solution

    I added an optional flag to the foreign key widget to enable it to use natural keys. I named the flag in keeping with other functions that accept natural keys in Django.

    Happy to add some more tests if anyone has suggestions, and could definitely change the behavior somewhat.

    Acceptance Criteria

    • I added tests for the natural key functions, and supporting changes to the test models
    • I updated the docstring for ForeignKeyWidget with the new parameter
    • I manually tested making a new resource and importing/exporting with the natural key widget which was successful, screenshot below: natural key import screen change
    opened by pokken-magic 6
  • (WIP) Flake8 + Black checks in GitHub actions

    (WIP) Flake8 + Black checks in GitHub actions

    Add flake8 configuration and flake8 + Black tests in GitHub actions as mentioned in #1223.

    This PR doesn't include actual code linting fixes, as we should discuss the exact configuration of Black/Flake8 first.

    opened by PetrDlouhy 1
  • Release 3 x

    Release 3 x

    This PR is a combination of other PRs which will form release 3.0.0

    opened by matthewhegarty 1
  • Resource._meta doesn't get params from parent's parent's Meta

    Resource._meta doesn't get params from parent's parent's Meta

    Describe the bug Resource._meta does get params from own Meta and from parent's Meta. But it doesn't get params from Meta of parent's parent's Meta

    To Reproduce Execute the following code:

    from import_export.resources import Resource
    
    class MyBaseResource(Resource):
        class Meta:
            # random parameter to show error
            batch_size = 123456789
    
    
    class MyResource(MyBaseResource):
        class Meta:
            pass
    
    
    class MyChildResource(MyResource):
        class Meta:
            pass
    
    print(f'MyResource: {MyResource()._meta.batch_size}')
    print(f'MyChildResource: {MyChildResource()._meta.batch_size}')
    

    Output:

    MyResource: 123456789 # right
    MyChildResource: 1000 # wrong, default value of `ResourceOptions.batch_size`
    

    Versions (please complete the following information):

    • Django Import Export: 2.7.0
    • Python 3.9
    • Django 4.0

    Expected behavior MyChildResource should get params from MyBaseResource

    Possible Solution import_export.resources.DeclarativeMetaclass should copy params from parent's _meta instead of Meta.

    bug 
    opened by ramazanFarrahov 1
  • Improve error information, include exact value and column

    Improve error information, include exact value and column

    It would be nice quality of life feature if we could return more detailed error info. Including exact value/column which raised it. Something similar to https://github.com/django-import-export/django-import-export/issues/1215

    End goal would be for row_errors to be changed to namedtuple or similar formatted the same as currently but with expanded info about exact value/column where error happened. Caller can than more easily format this and return back in same file by appending it or new file or show in template...

    As far i can see this return line import_obj would need modification.

    I would open pr for this. Any hints before i start?

    enhancement question 
    opened by MarkoBox 1
Releases(2.7.1)
Jet Bridge (Universal) for Jet Admin – API-based Admin Panel Framework for your application

Jet Bridge for Jet Admin – Admin panel framework for your application Description About Jet Admin: https://about.jetadmin.io Live Demo: https://app.je

Jet Admin 1.1k Jan 16, 2022
aiohttp admin is generator for admin interface based on aiohttp

aiohttp admin is generator for admin interface based on aiohttp

Mykhailo Havelia 9 Dec 1, 2021
A cool, modern and responsive django admin application based on bootstrap 5

django-baton A cool, modern and responsive django admin application based on bootstrap 5 Documentation: readthedocs Live Demo Now you can try django-b

Otto srl 531 Jan 18, 2022
📱 An extension for Django admin that makes interface mobile-friendly. Merged into Django 2.0

Django Flat Responsive django-flat-responsive is included as part of Django from version 2.0! ?? Use this app if your project is powered by an older D

elky 248 Dec 3, 2021
An improved django-admin-tools dashboard for Django projects

django-fluent-dashboard The fluent_dashboard module offers a custom admin dashboard, built on top of django-admin-tools (docs). The django-admin-tools

django-fluent 302 Dec 30, 2021
A Django app for easily adding object tools in the Django admin

Django Object Actions If you've ever tried making admin object tools you may have thought, "why can't this be as easy as making Django Admin Actions?"

Chris Chang 424 Jan 15, 2022
Disable dark mode in Django admin user interface in Django 3.2.x.

Django Non Dark Admin Disable or enable dark mode user interface in Django admin panel (Django==3.2). Installation For install this app run in termina

Artem Galichkin 3 Jan 10, 2022
Helpers to extend Django Admin with data from external service with minimal hacks

django-admin-data-from-external-service Helpers to extend Django Admin with data from external service with minimal hacks Live demo with sources on He

Evgeniy Tatarkin 5 Nov 30, 2021
Extends the Django Admin to include a extensible dashboard and navigation menu

django-admin-tools django-admin-tools is a collection of extensions/tools for the default django administration interface, it includes: a full feature

Django Admin Tools 656 Jan 18, 2022
Modern responsive template for the Django admin interface with improved functionality. We are proud to announce completely new Jet. Please check out Live Demo

Django JET Modern template for Django admin interface with improved functionality Attention! NEW JET We are proud to announce completely new Jet. Plea

Geex Arts 3.2k Jan 10, 2022
Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap.

Xadmin Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap. Liv

差沙 4.6k Jan 16, 2022
A jazzy skin for the Django Admin-Interface (official repository).

Django Grappelli A jazzy skin for the Django admin interface. Grappelli is a grid-based alternative/extension to the Django administration interface.

Patrick Kranzlmueller 3.2k Jan 15, 2022
A Django admin theme using Twitter Bootstrap. It doesn't need any kind of modification on your side, just add it to the installed apps.

django-admin-bootstrapped A Django admin theme using Bootstrap. It doesn't need any kind of modification on your side, just add it to the installed ap

null 1.6k Dec 14, 2021
django's default admin interface made customizable. popup windows replaced by modals. :mage: :zap:

django-admin-interface django-admin-interface is a modern responsive flat admin interface customizable by the admin itself. Features Beautiful default

Fabio Caccamo 1k Jan 12, 2022
Extendable, adaptable rewrite of django.contrib.admin

django-admin2 One of the most useful parts of django.contrib.admin is the ability to configure various views that touch and alter data. django-admin2

Jazzband 1.1k Dec 27, 2021
Modern theme for Django admin interface

Django Suit Modern theme for Django admin interface. Django Suit is alternative theme/skin/extension for Django administration interface. Project home

Kaspars Sprogis 2.1k Jan 16, 2022
:honey_pot: A fake Django admin login screen page.

django-admin-honeypot django-admin-honeypot is a fake Django admin login screen to log and notify admins of attempted unauthorized access. This app wa

Derek Payton 832 Jan 14, 2022
"Log in as user" for the Django admin.

django-loginas About "Login as user" for the Django admin. loginas supports Python 3 only, as of version 0.4. If you're on 2, use 0.3.6. Installing dj

Stavros Korokithakis 292 Dec 26, 2021
Visually distinguish environments in Django Admin

django-admin-env-notice Visually distinguish environments in Django Admin. Based on great advice from post: 5 ways to make Django Admin safer by hakib

Yuri Shikanov 236 Jan 3, 2022