GBIF NODES9, GB14, Amsterdam, October 15 2007GBIF NODES9, GB14, Amsterdam, October 15 2007

Cover Cover slideslideTAPIR

PyWrapper

The third generation PyWrapper

October 15, 2007NODES9, GB14, Amsterdam

Dag Terje Filip EndresenNordic Gene Bank (Sweden)Bioversity International (Italy)

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 2

TOPICSTOPICS

TAPIR PyWrapper

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 3

TAPIR

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 4

TDWG Access Protocol for Information Retrieval - TDWG Access Protocol for Information Retrieval - TAPIRTAPIR

[http://www.tdwg.org/activities/tapir/]

TAPIR specifies a standardised, stateless, HTTP transmittable, XML-based request and response protocol for accessing structured data.

TAPIR combines and extends features of the BioCASE and DiGIR protocols.

The work on the TAPIR protocol was initiated at the 2004 TDWG conference in New Zealand.

The mandate of TAPIR is to unify the protocol of the DiGIR and BioCASE wrappers, which is most common in current implementations for sharing of biodiversity datasets.

Three alternative TAPIR reference implementations are developed: PyWrapper [http://www.pywrapper.org] TapirLink [http://sourceforge.net/projects/digir/] TapirDotNET [http://sourceforge.net/projects/tapirdotnet/]

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 5

Biodiversity data exchange tools

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 6

PyWrapper3PyWrapper3

PyWrapper3 is based on the previous PyWrapper version 1 and 2 from the BioCASE project.

PyWrapper3 comes with a small python application server (CherryPy) that can be run as a standalone web server.

The recommended configuration is to run PyWrapper behind an Apache web server.

PyWrapper3 includes a LSID resolver sevice.

The PyWrapper3 middleware can be used to implement distributed information systems by using the TAPIR data exchange protocol.

All datasources wrapped by PyWrapper can be accessed in a uniform way, no matter what data structure they have.

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 7

Required configuration:

Web server: Any CGI compliant web server: Apache, IIS etc. (The built in CherryPy web server can also be used).

Database: major databases are supported, including MySQL, Oracle, SQLServer, Sybase, Access, PostgreSQL. Theoretically any database with a Python library should work.

Python PyWrapper is developed with the Python programming language. (Install version 2.3 or later).

[http://trac.pywrapper.org/pywrapper/wiki/Downloads][With SubVersion svn://svn.pywrapper.org/pywrapper]

PyWrapper Provider SoftwarePyWrapper Provider Software

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 8

InstallationInstallation

PyWrapper should run on any operating system that supports Python [http://www.python.org/].

Python runs on Windows, Linux/Unix, Mac OS X, OS/2, Amiga, Palm Handhelds, and Nokia mobile phones. Python has also been ported to the Java and .NET virtual machines.

Python is distributed under an OSI-approved open source license that makes it free to use, even for commercial products.

You will also need to install these Python libraries:- PyXML- mxDateTime - lxml for libxml2- pyPgSQL for PostgreSQL or MySQLdb for MySQL.

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 9

InstallationInstallation

Step 1: Download the latest PyWrapper3 installer.

Step 2: Uncompress to a folder of your choice.- Example: “/usr/local/pywrapper/”- Example: “C:\pywrapper\”

Step 3. Installation …

Local installation: If you have a Subversion client installed, you may use the automatic installer.(Local Python and libraries is installed to your pywrapper folder.)

promt$ svn export svn://svn.pywrapper.org:80/pywrapper/trunk pywrapper

promt$ cd pywrapper/toolspromt$ /bin/sh install.sh

This will require that you have a bash shell, and probably that you have a Unix line system like e.g. FreeBSD, Linux or Mac OsX…

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 10

System installation:

Make sure you have Python 2.4 or later installed.

Install the Python libraries:- PyXML- mxDateTime - lxml for libxml2- pyPgSQL if you will use PostgreSQL- MySQLdb if you will use MySQL

$promt setup.py install (Unix-like systems).For Windows systems, execute the installer.

Execute: pywrapper/tools/adapt_installation.py(file permissions and shebang line in the python

scripts)

InstallationInstallation

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 11

After successful installation you will need to configure your data provider. Follow the instructions from the PyWrapper documentation web page to configure

Data sources. If you provide more datasets or several databases they will be configured as individual data sources.

Database connection. So the software can access your database.

Database structure. Define the relevant tables, the primary keys and foreign keys.

Data model. Map your database model to the standard represented by the XML Schemas you choose.

PyWrapper configurationPyWrapper configuration

[http://trac.pywrapper.org/pywrapper/wiki/Documentation][http://trac.pywrapper.org/pywrapper/wiki/DraftInstallationGuide]

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 12

Start standalone server:

run ./start_server.py (default port is 8080)

promt$ cd webapp/ promt$ ./start_server.py 8088

Configure PyWrapper v3.1.0dev CherryPy server using /somewhere/pywrapper/config/webapp.cfg

mounting PyWrapper webapps to /pywrapperrun server on port 8088http://localhost:8088/pywrapperstart engine...

http://localhost:8088/pywrapper

/pywrapper/config/config.ini

Start PyWrapperStart PyWrapper

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 13

PyWrapper comes with a graphical web based configuration tool

ConfigurationConfiguration

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 14

Install third party software[http://localhost/pywrapper/utilities/testlibs.cgi ]

Follow the links from the Library test page.

The column for installed version will display the installed version after successful installation.

BioCASEBioCASE

To update the PyWrapper software:

Download the new release. Unzip to a temporary folder. Execute the setup.py and follow the

instructions.

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 15

Example of a service Example of a service requestrequest

All exchanged data is formatted with XML tags.

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 16

Example of a service Example of a service responseresponse

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 17

Example 2, service requestExample 2, service request

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 18

Example 2, service responseExample 2, service response

singer:/sourcenamesinger:/taxonomy/genussinger:/taxonomy/speciessinger:/taxonomy/subspeciessinger:/holding/IDsinger:/holding/namesinger:/origin/collecting/

countrysourcesinger:/origin/collecting/

countrysourceIDsinger:/status/biologicalstatussinger:/status/biologicalstatusID

...

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 19

Taxonomic Database Working GroupTaxonomic Database Working Group

Darwin Core 2 - Element definitions designed to support the sharing and integration of primary biodiversity data". [http://wiki.tdwg.org/twiki/bin/view/DarwinCore/WebHome], [http://darwincore.calacademy.org/]

Access to Biological Collection Data (ABCD) 2.06 - An evolving comprehensive standard for the access to and exchange of data about specimens and observations (a.k.a. primary biodiversity data)“.[http://www.tdwg.org/activities/abcd/],[http://www.bgbm.org/TDWG/CODATA/Schema/]

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 20

PyWrapper/BioCASE PyWrapper/BioCASE [http://www.pywrapper.org/][http://www.pywrapper.org/]

PyWrapper runs on MS Windows, Mac OS X, Linux, BSD, Solaris...

PyWrapper works with many different databases, PostgreSQL, MySQL, Oracle, MS Access, MS SQL Server....

PyWrapper works with UNICODEאבדו ضاإطقكغب ששچپچ

PyWrapper is OpenSource

PyWrapper is developed in the Python programming language

CVS

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 21

Life Science IDentifiers

LSID is a digital name tag. LSIDs are GUIDs, Global Unique Identifiers.

[http://lsid.sourceforge.net/]

Structure urn:lsid:authority:namespace:object:revision Example (fictive) urn:lsid:eurisco.org:accession:H451269

The LSID concept introduces a straightforward approach to naming and identifying data resources stored in multiple, distributed data stores.

LSID defines a simple, common way to identify and access biologically significant data; whether that data is stored in files, relational databases, in applications, or in internal or public data sources, LSID provides a naming standard to support interoperability.

Developed by OMG-LSR and W3C, implemented by IBM.

W3C :: LSIDW3C :: LSID

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 22

Distributed BioCASE/PyWrapper networkDistributed BioCASE/PyWrapper network

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 23

Special thanks toSpecial thanks to

Bioversity International [http://www.bioversityinternational.org]

GBIF, Global Biodiversity Information Facility [http://www.gbif.org]

BioCASE, The Biological Collection Access Service for Europe. [http://www.biocase.org]

TDWG, Taxonomic Database Working Group [http://www.tdwg.org]

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 24

Germplasm data indexing toolsGermplasm data indexing tools

We have recently built data indexing methodologies for access to germplasm data with BioCASE/PyWrapper.

This is planned to build a Global Accession Information Network (GAIN).

Development in close cooperation with GBIF, which themselves index basic biodiversity data from a similar approach.

[http://chm.grinfo.net/index.php]

TAPIR PyWrapper, October 15 2007, NODES9, GB14, AmsterdamTAPIR PyWrapper, October 15 2007, NODES9, GB14, Amsterdam 25

Thank you for listening!