django-import-export documentation · 2019-04-02 · django-import-export documentation, release...

django-import-export DocumentationRelease 0.1.4

Bojan Mihelac

August 27, 2013

CONTENTS

i

django-import-export Documentation, Release 0.1.4

django-import-export is a Django application and library for importing and exporting data with included admin inte-gration.

Features:

• support multiple formats (Excel, CSV, JSON, ... and everything else that tablib support)

• admin integration for importing

• preview import changes

• admin integration for exporting

• export data respecting admin filters

CONTENTS 1

https://github.com/kennethreitz/tablib


2 CONTENTS

CHAPTER

ONE

USER GUIDE

1.1 Installation

django-import-export is on the Python Package Index (PyPI), so it can be installed with standard Python tools like pipor easy_install:

$pip install django-import-export

1.2 Configuration

The only mandatory configuration is adding import_export to your INSTALLED_APPS. This isn’t necessary, ifadmin integration is not used.

1.3 Getting started

For example purposes, we’ll be use simplified book app, here is our core.models.py:

class Author(models.Model):name = models.CharField(max_length=100)

def __unicode__(self):return self.name

class Category(models.Model):name = models.CharField(max_length=100)


class Book(models.Model):name = models.CharField(’Book name’, max_length=100)author = models.ForeignKey(Author, blank=True, null=True)author_email = models.EmailField(’Author email’, max_length=75, blank=True)imported = models.BooleanField(default=False)published = models.DateField(’Published’, blank=True, null=True)price = models.DecimalField(max_digits=10, decimal_places=2, null=True,

blank=True)

3


categories = models.ManyToManyField(Category, blank=True)


1.3.1 Creating import-export resource

To integrate django-import-export with Book model, we will create resource class that will describe how this resourcecan be imported or exported.

from import_export import resourcesfrom core.models import Book

class BookResource(resources.ModelResource):

class Meta:model = Book

1.3.2 Exporting data

Now that we have defined resource class, we can export books:

>>> dataset = BookResource().export()>>> print dataset.csvid,name,author,author_email,imported,published,price,categories2,Some book,1,,0,2012-12-05,8.85,1

1.3.3 Customize resource options

By default ModelResource introspects model fields and creates import_export.fields.Field attributewith appopriate widget for each field.

To affect which model fields will be included in import-export resource, use fields option to whitelist fields orexclude option for to blacklist fields:


class Meta:model = Bookexclude = (’imported’, )

When defining ModelResource fields it is possible to follow model relationships:


class Meta:model = Bookfields = (’author__name’,)

Note: Following relationship fields sets field as readonly, meaning this field will be skipped when importing data.

See Also:

4 Chapter 1. User Guide


Resources

1.3.4 Declaring fields

It is possible to override resource fields to change some of it’s options:

from import_export import fields

class BookResource(resources.ModelResource):published = fields.Field(column_name=’published_date’)


Other fields, that are not existing in target model may be added:


class BookResource(resources.ModelResource):myfield = fields.Field(column_name=’myfield’)


See Also:

Fields Available field types and options.

1.3.5 Advanced data manipulation

Not all data can be easily pull off an object/model attribute. In order to turn complicated data model into a (generallysimpler) processed data structure, dehydrate_<fieldname> method should be defined:


class BookResource(resources.ModelResource):full_title = fields.Field()


def dehydrate_full_title(self, book):return ’%s by %s’ % (book.name, book.name.author)

1.3.6 Customize widgets

ModelResource creates field with default widget for given field type. If widget should be initialized with differentarguments, set widgets dict.

In this example widget for published field is overriden to use different date format. This format will be used bothfor importing and exporting resource.



1.3. Getting started 5


widgets = {’published’: {’format’: ’%d.%m.%Y’},}

See Also:

Widgets available widget types and options.

1.3.7 Importing data

Let’s import data:

>>> import tablib>>> from import_export import resources>>> from core.models import Book>>> book_resource = resources.modelresource_factory(model=Book)()>>> dataset = tablib.Dataset([’’, ’New book’], headers=[’id’, ’name’])>>> result = book_resource.import_data(dataset, dry_run=True)>>> print result.has_errors()False>>> result = book_resource.import_data(dataset, dry_run=False)

In 4th line we use modelresource_factory to create default ModelResource. ModelResource class createdthis way is equal as in Creating import-export resource.

In 5th line Dataset with subset of Book fields is created.

In rest of code we first pretend to import data with dry_run set, then check for any errors and import data.

See Also:

Import data workflow for detailed import workflow descripton and customization options.

Deleting data

To delete objects during import, implement for_delete method on resource class.

Example resource with delete field:

class BookResource(resources.ModelResource):delete = fields.Field(widget=widgets.BooleanWidget())

def for_delete(self, row, instance):return self.fields[’delete’].clean(row)


Import of this resource will delete model instances for rows that have column delete set to 1.

1.3.8 Admin integration

Admin integration is achived by subclassing ImportExportModelAdmin or one of mixins:

from import_export.admin import ImportExportModelAdmin



class BookAdmin(ImportExportModelAdmin):# resouce_class = BookResourcepass

See Also:

Admin available mixins and options.

1.4 Import data workflow

This document describes import data workflow, with hooks that enable customization of import process.

1.4.1 import_data method arguments

import_data method of import_export.resources.Resource class is responsible for import data fromgiven dataset.

import_data expect following arguments:

dataset REQUIRED. should be Tablib Dataset object with header row.

dry_run If True, import should not change database. Default is False.

raise_errors If True, import should raise errors. Default is False, which means that eventual errors andtraceback will be saved in Result instance.

1.4.2 import_data method workflow

1. import_data intialize new import_export.results.Result instance. Result instance holds er-rors and other information gathered during import.

2. InstanceLoader responsible for loading existing instances is intitalized.

Different InstanceLoader class can be specified with instance_loader_class option ofimport_export.resources.ResourceOptions.

import_export.instance_loaders.CachedInstanceLoader can be used to reduce number ofdatabase queries.

See import_export.instance_loaders for available implementations.

3. Process each row in dataset

(a) get_or_init_instance method is called with current InstanceLoader and current row return-ing object instance and Boolean variable that indicates if object instance is new.

get_or_init_instance tries to load instance for current row or calls init_instance to initobject if object does not exists yet.

Default ModelResource.init_instance initialize Django Model without arguments. You canoverride init_instance method to manipulate how new objects are initialized (ie: to set default val-ues).

(b) for_delete method is called to determine if current instance should be deleted:

i. current instance is deleted

OR

1.4. Import data workflow 7

http://docs.python-tablib.org/en/latest/api/#dataset-object


i. import_obj method is called with current object instance and current row.

import_obj loop through all Resource fields, skipping many to many fields and callsimport_field for each. (Many to many fields require that instance have a primary key, this iswhy assigning them is postponed, after object is saved).

import_field calls field.save method, if field has both attribute and field column_nameexists in given row.

ii. save_instance method is called.

save_instance receives dry_run argument and actually saves instance only when dry_run isFalse.

save_instance calls two hooks methods that by default does not do anything but can be overridento customize import process:

• before_save_instance

• after_save_instance

Both methods receive instance and dry_run arguments.

iii. save_m2m method is called to save many to many fields.

(c) RowResult is assigned with diff between original and imported object fields as well as importtype(new, updated).

If exception is raised inside row processing, and raise_errors is False (default), traceback isappended to RowResult.

4. result is returned.

1.4.3 Transaction support

If transaction support is enabled, whole import process is wrapped inside transaction and rollbacked or committedrespectively. All methods called from inside of import_data (create / delete / update) receive False for dry_runargument.

1.5 Example app

cd tests && ./manage.py runserver

Username and password for admin are ‘admin’, ‘password’.

1.6 Settings

IMPORT_EXPORT_USE_TRANSACTIONS Global setting controls if resource importing should use database trans-actions. Default is False.



1.7 TODO

1.8 Contributing

1.8.1 Code guidelines

• As most projects, we try to follow PEP8 as closely as possible

• Most pull requests will be rejected without proper unit testing

1.9 Change Log

1.9.1 0.1.4

• Use field_name instead of column_name for field dehydration, FIX #36

• Handle OneToOneField, FIX #17 - Exception when attempting access something on the related_name.

• FIX #23 - export filter not working

1.9.2 0.1.3

• Fix packaging

• DB transactions support for importing data

1.9.3 0.1.2

• support for deleting objects during import

• bug fixes

• Allowing a field to be ‘dehydrated’ with a custom method

• added documentation

1.9.4 0.1.1

• added ExportForm to admin integration for choosing export file format

• refactor admin integration to allow better handling of specific formats supported features and better handling ofreading text files

• include all avialable formats in Admin integration

• bugfixes

1.9.5 0.1.0

• Refactor api

1.7. TODO 9

CHAPTER

TWO

API DOCUMENTATION

2.1 Resources

2.1.1 Resource

class import_export.resources.ResourceResource defines how objects are mapped to their import and export representations and handle importing andexporting data.

after_delete_instance(instance, dry_run)Override to add additional logic.

after_save_instance(instance, dry_run)Override to add additional logic.

before_delete_instance(instance, dry_run)Override to add additional logic.

before_save_instance(instance, dry_run)Override to add additional logic.

export(queryset=None)Exports a resource.

for_delete(row, instance)Returns True if row importing should delete instance.

Default implementation returns False. Override this method to handle deletion.

get_diff(original, current, dry_run=False)Get diff between original and current object when import_data is run.

dry_run allows handling special cases when object is not saved to database (ie. m2m relationships).

get_diff_headers()Diff representation headers.

classmethod get_field_name(field)Returns field name for given field.

get_fields()Returns fields in export_order order.

import_data(dataset, dry_run=False, raise_errors=False, use_transactions=None)Imports data from dataset.

11


use_transactions If True import process will be processed inside transaction. If dry_run is set,or error occurs, transaction will be rolled back.

import_obj(obj, data)

save_m2m(obj, data, dry_run)Saves m2m fields.

Model instance need to have a primary key value before a many-to-many relationship can be used.

2.1.2 ModelResource

class import_export.resources.ModelResourceModelResource is Resource subclass for handling Django models.

classmethod widget_from_django_field(f, default=<class ‘import_export.widgets.Widget’>)Returns the widget that would likely be associated with each Django type.

classmethod widget_kwargs_for_field(field_name)Returns widget kwargs for given field_name.

2.1.3 ResourceOptions (Meta)

class import_export.resources.ResourceOptionsThe inner Meta class allows for class-level configuration of how the Resource should behave. The followingoptions are available:

•fields - Controls what introspected fields the Resource should include. A whitelist of fields.

•exclude - Controls what introspected fields the Resource should NOT include. A blacklist of fields.

•model - Django Model class. It is used to introspect available fields.

•instance_loader_class - Controls which class instance will take care of loading existing objects.

•import_id_fields - Controls which object fields will be used to identify existing instances.

•export_order - Controls export order for columns.

•widgets - dictionary defines widget kwargs for fields.

•use_transactions - Controls if import should use database transactions. Default value is Nonemeaning settings.IMPORT_EXPORT_USE_TRANSACTIONS will be evaluated.

2.2 Fields

class import_export.fields.Field(attribute=None, column_name=None, widget=None, read-only=False)

Field represent mapping between object field and representation of this field.

attribute string of either an instance attribute or callable off the object.

column_name let you provide how this field is named in datasource.

widget defines widget that will be used to represent field data in export.

readonly boolean value defines that if this field will be assigned to object during import.

clean(data)Takes value stored in the data for the field and returns it as appropriate python object.

12 Chapter 2. API documentation


export(obj)Returns value from the provided object converted to export representation.

get_value(obj)Returns value for this field from object attribute.

save(obj, data)Cleans this field value and assign it to provided object.

2.3 Widgets

class import_export.widgets.WidgetWidget takes care of converting between import and export representations.

Widget objects have two functions:

•converts object field value to export representation

•converts import value and converts it to appropriate python representation

clean(value)Returns appropriate python objects for import value.

render(value)Returns export representation of python value.

class import_export.widgets.IntegerWidgetWidget for converting integer fields.

class import_export.widgets.DecimalWidgetWidget for converting decimal fields.

class import_export.widgets.CharWidgetWidget for converting text fields.

class import_export.widgets.BooleanWidgetWidget for converting boolean fields.

class import_export.widgets.DateWidget(format=None)Widget for converting date fields.

Takes optional format parameter.

class import_export.widgets.ForeignKeyWidget(model, *args, **kwargs)Widget for ForeignKey model field that represent ForeignKey as integer value.

Requires a positional argument: the class to which the field is related.

class import_export.widgets.ManyToManyWidget(model, *args, **kwargs)Widget for ManyToManyField model field that represent m2m field as comma separated pk values.

Requires a positional argument: the class to which the field is related.

2.4 Instance loaders

class import_export.instance_loaders.BaseInstanceLoader(resource, dataset=None)Base abstract implementation of instance loader.

2.3. Widgets 13


class import_export.instance_loaders.ModelInstanceLoader(resource, dataset=None)Instance loader for Django model.

Lookup for model instance by import_id_fields.

class import_export.instance_loaders.CachedInstanceLoader(*args, **kwargs)Loads all possible model instances in dataset avoid hitting database for every get_instance call.

This instance loader work only when there is one import_id_fields field.

2.5 Admin

import_export.admin.DEFAULT_FORMATS = (<class ‘import_export.formats.base_formats.CSV’>, <class ‘import_export.formats.base_formats.XLS’>, <class ‘import_export.formats.base_formats.TSV’>, <class ‘import_export.formats.base_formats.ODS’>, <class ‘import_export.formats.base_formats.JSON’>, <class ‘import_export.formats.base_formats.YAML’>, <class ‘import_export.formats.base_formats.HTML’>)import / export formats

class import_export.admin.ExportMixinExport mixin.

change_list_template = ‘admin/import_export/change_list_export.html’template for change_list view

export_template_name = ‘admin/import_export/export.html’template for export view

formats = (<class ‘import_export.formats.base_formats.CSV’>, <class ‘import_export.formats.base_formats.XLS’>, <class ‘import_export.formats.base_formats.TSV’>, <class ‘import_export.formats.base_formats.ODS’>, <class ‘import_export.formats.base_formats.JSON’>, <class ‘import_export.formats.base_formats.YAML’>, <class ‘import_export.formats.base_formats.HTML’>)available import formats

get_export_formats()Returns available import formats.

get_export_queryset(request)Returns export queryset.

Default implementation respects applied search and filters.

resource_class = Noneresource class

to_encoding = ‘utf-8’export data encoding

class import_export.admin.ImportExportMixinImport and export mixin.

change_list_template = ‘admin/import_export/change_list_import_export.html’template for change_list view

class import_export.admin.ImportExportModelAdmin(model, admin_site)Subclass of ModelAdmin with import/export functionality.

class import_export.admin.ImportMixinImport mixin.

change_list_template = ‘admin/import_export/change_list_import.html’template for change_list view

formats = (<class ‘import_export.formats.base_formats.CSV’>, <class ‘import_export.formats.base_formats.XLS’>, <class ‘import_export.formats.base_formats.TSV’>, <class ‘import_export.formats.base_formats.ODS’>, <class ‘import_export.formats.base_formats.JSON’>, <class ‘import_export.formats.base_formats.YAML’>, <class ‘import_export.formats.base_formats.HTML’>)available import formats

from_encoding = ‘utf-8’import data encoding



get_import_formats()Returns available import formats.

import_action(request, *args, **kwargs)Perform a dry_run of the import to make sure the import will not result in errors. If there where no error,save the user uploaded file to a local temp file that will be used by ‘process_import’ for the actual import.

import_template_name = ‘admin/import_export/import.html’template for import view

process_import(request, *args, **kwargs)Perform the actual import action (after the user has confirmed he wishes to import)

resource_class = Noneresource class

2.6 Results

2.6.1 Result

class import_export.results.Result(*args, **kwargs)

2.6. Results 15

PYTHON MODULE INDEX

iimport_export.admin, ??import_export.instance_loaders, ??

17

django-import-export documentation · 2019-04-02 · django-import-export documentation, release...

Documents