workshop matlab bits database

WMDB 2008BITS Time Series Data Base

Emmanuele [email protected]

Supporto Informatico per l’Area RicercheBanca d’Italia

21 January 2008

E. Somma (SIA-BdI) WMDB 2008 21/01/2008 1/28

Summary

1 tsdb The Time Series Database with Vintage SupportLoading DataGetting informations

2 tsdb Administration


tsdb The Time Series Database with Vintage Support




tsdb Time Series Database Object

Single object for all Database operations (tsdb)

Very simple command interface

Saves data, release date, series name and every metadata you wish

Issues (standard) SQL Data Manipulation commands to RDBMS.

Issue also Data Definition commands to fully create database instanceor grant access to user (db root authorizations required)

Fully support free MYSQL database, compatible to other DB (DB2and Oracle tested).

Native access to JDBC drivers (more speed) or ODBC Connectors(more compatibility)


tsdb Basic Usage

>> db = tsdb( database, user, password)>> ts = tsload(db, ’SERIENAME’)>> [...]>> tsstore(db, ts)


tsdb What is a Release?

A RELEASE is a value representing a lapse in time at your wish (i.e. whenyou downloaded the series).The field containing the release information is named release and is inthe columns metadata.Actual TSDB implementation recognize release as MATLAB datenumvalue.Even the container (tsmat) has a release metadata, it NOT reports therelease of the series contained but only the release of the collection instead.The series (column) release is saved into the DB with tstore operationand loaded with tsload whereas collection release is not saved andloaded. Only if isn’t defined a release for a series/column tsstoreoperation uses collection release as series release.


tsdb Collection and Series Releases

>>

tstab(ts)tsStart: 1980 1tsRelease: 25-Jul-2007

COLLECTION RELEASE

Frequency: 12 (monthly)Date

SERIES RELEASE

T1 T1 T1 T117/07/07 18/07/07 19/07/07 20/07/07

1980-M-001 0.6822 0.1509 0.8600 0.49661980-M-002 0.3028 0.6979 0.8537 0.89981980-M-003 0.5417 0.3784 0.5936 0.8216



>> tstab(ts)

tsStart: 1980 1tsRelease: 25-Jul-2007

COLLECTION RELEASE


SERIES RELEASE

T1 T1 T1 T117/07/07 18/07/07 19/07/07 20/07/07

1980-M-001 0.6822 0.1509 0.8600 0.49661980-M-002 0.3028 0.6979 0.8537 0.89981980-M-003 0.5417 0.3784 0.5936 0.8216



>> tstab(ts)tsStart: 1980 1tsRelease: 25-Jul-2007

COLLECTION RELEASE


SERIES RELEASE

T1 T1 T1 T117/07/07 18/07/07 19/07/07 20/07/07

1980-M-001 0.6822 0.1509 0.8600 0.49661980-M-002 0.3028 0.6979 0.8537 0.89981980-M-003 0.5417 0.3784 0.5936 0.8216



>> tstab(ts)tsStart: 1980 1tsRelease: 25-Jul-2007 COLLECTION RELEASEFrequency: 12 (monthly)Date

SERIES RELEASE

T1 T1 T1 T117/07/07 18/07/07 19/07/07 20/07/07

1980-M-001 0.6822 0.1509 0.8600 0.49661980-M-002 0.3028 0.6979 0.8537 0.89981980-M-003 0.5417 0.3784 0.5936 0.8216



>> tstab(ts)tsStart: 1980 1tsRelease: 25-Jul-2007

COLLECTION RELEASE

Frequency: 12 (monthly)Date SERIES RELEASE

T1 T1 T1 T117/07/07 18/07/07 19/07/07 20/07/07

1980-M-001 0.6822 0.1509 0.8600 0.49661980-M-002 0.3028 0.6979 0.8537 0.89981980-M-003 0.5417 0.3784 0.5936 0.8216


tsdb Using Single and Multiple Parameters

Some TSDB operations may be used with a single parameter as well withmultiple parameters.Single parameter could be a simple string, numbers or series. Multipleparameters have to be a cell array containing strings, numbers or series.

>> ts = tsload(db, ’SERIES’)>> ts = tsload(db, { ’SERIES1’ ’SERIES2’})>> ts = tsload(db, { ’SERIES1’ ’SERIES2’}, 733283)>> ts = tsload(db, { ’SERIES1’ ’SERIES2’}, {733283 733284})


tsdb A complete example

Follow instructions to setup free database software on your machine

Follow instructions to setup Java connector on your machine

Connect as database administrator and create a TSDB instance

Connect as BITS user and test installation

Play freely

Return to work


tsdb Setup free DBMS (Mysql)

Downloads and click on XAMPP.EXE(from file://stux20/pccommon/xampp)

Unzip in D:/

Done (Your administrator user is root. No password.)


tsdb Setup Java

Downloads mysql-connector-java-5.0.6-bin.jar(from file://stux20/pccommon/xampp)

Save in D:/xampp/mysql/

>> javaaddpath D:/xampp/mysql

>> clear java


tsdb DEMO: Create demo database

From Matlab prompt issue the commands:

>> dbROOT = tsdb(’J:mysql’,’root’,’’)

>> tsdbdemo(dbROOT,’create’)

>> close(dbROOT)


tsdb DEMO: Check database

>> db = tsdb(’J:bitsdemo’,’BITS’,’’)BITSDB Database bitsdemo (ver. 1.0)User: BITS

On Host: localhostBackup Dir: C:\Documents[...]\h856605\Application Data\BITSDB

Driver: jdbc:mysql://localhost/bitsdemo

>> namelist=tsseries(db)

namelist = ’ITRETTOTF’’ITIPINTMG’’BDESPISDH’’BDEU2001A’’BDM3C...B’’FRHCONMGD’’EAGDP...D’’ITGDP...D’

OK. Database ’bitsdemo’ is up and running.E. Somma (SIA-BdI) WMDB 2008 21/01/2008 13/28

Loading Data




tsdb DEMO: Loading Data

>> [ tsM, tsA ] = tsload(db,namelist)tsM =

matdata: [258x6 double]start_year: 1986

[...]Size: [258 6]Start: 1986 1

End: 2007 6Labels: ITRETTOTF ITIPINTMG BDESPISDH BDEU2001A BDM3C...B

tsA =matdata: [89x2 double]

start_year: 1985[...]

Size: [89 2]Start: 1985 1

End: 2007 1Labels: EAGDP...D ITGDP...D


tsdb DEMO: Loading Data/2

>> tstab(tsM)tsStart: 1986 1tsRelease: 19-Jul-2007Frequency: 12 (monthly)Date

ITRETTOTF ITIPINTMG BDESPISDH BDEU2001A BDM3C...B FRHCONMGD12/07/07 12/07/07 12/07/07 12/07/07 12/07/07 12/07/07

1986-M-001 39.1000 78.8519 82.3500 22398.1600 601.8000 12.07801986-M-002 35.7600 79.5136 95.3100 21864.3700 602.9000 11.90101986-M-003 42.5700 79.4736 101.4700 22359.8200 605.3000 12.23301986-M-004 41.1700 79.2002 96.9700 25555.3900 607.4000 12.4070[...]2007-M-004 106.6000 95.8000 91.1000 78810.0000 1700.0000 20.87702007-M-005 NaN 96.6000 91.3000 79077.0000 1711.8000 20.71102007-M-006 NaN NaN NaN NaN NaN NaN


tsdb DEMO: Loading Releases

>> r = tsreleases(db,’BDESPISDH’);>> ts = tsload(db,’BDESPISDH’,r)ts =

matdata: [258x25 double][...]

Size: [258 25]Start: 1986 1End: 2007 6

Labels: BDESPISDH BDESPISDH ...


tsdb DEMO: Loading Data/2

>> tstab(ts)tsStart: 1986 1tsRelease: 06-Sep-2007Frequency: 12 (monthly)Date

BDESPISDH BDESPISDH BDESPISDH28/02/07 08/03/07 09/03/07

1986-M-001 82.3500 82.3500 82.35001986-M-002 95.3100 95.3100 95.31001986-M-003 101.4700 101.4700 101.4700[...]2006-M-012 89.3000 89.3000 91.60002007-M-001 NaN NaN 91.20002007-M-002 NaN NaN NaN2007-M-003 NaN NaN NaN


Getting informations




tsinfo

res = tsinfo(db,series,[operation, [arg, ... ] ] ...)op result = res{#idx}

== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


tsinfo

res = tsinfo(db,series,[operation, [arg, ... ] ] ...)op result = res{#idx}== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}

== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


tsinfo


== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}

== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)

== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


tsinfo


== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)

== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}

== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


tsinfo


== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}

== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}

== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


tsinfo


== NSERIES - Number of series in DB ==>> res = tsinfo(db, nan, ’nseries’)>> nseries = res{2}== SERIES - Names of the series in DB ==>> series = tsinfo(db,nan, ’series’)== NRELEASES - Number of releases in series ==>> res = tsinfo(db, {series}, ’nreleases’)>> nrels = res{1,1}>> nerls_serie1 = nrels{1}== RELEASES - Set of release dates for a given series ==>> rels = tsinfo(db, {series}, ’releases’)>> rels_serie1 = rels{1}

== ORDERBYFREQ - returns series ordered by frequency ==>> [ordered, freqs] = tsinfo(db, {series}, ’orderbyfreq’)


Connection status

>> ping(db)DatabaseProductName: ’MySQL’

DatabaseProductVersion: ’5.0.41-community-nt’JDBCDriverName: ’MySQL-AB JDBC Driver’

JDBCDriverVersion: [1x103 char]MaxDatabaseConnections: 0

CurrentUserName: ’BITS@localhost’DatabaseURL: ’jdbc:mysql://localhost/bitsdemo’

AutoCommitTransactions: ’True’

>>isconnection(db)ans =

1


tsdb Administration




tsadmin(dbROOT,’setup’,...)

Create USER if doesn’t exists.Requests password interactivelyor use empty password if DBNAME begins with ’*’

Create database bits<DBNAME>

Setup TSDB structure in ’bits<DBNAME>’

Example:

>> tsadmin(dbROOT,’setup’, ’BITS’,’test’)>> tsadmin(dbROOT,’setup’,’*BITS’,’test’)


tsadmin(dbROOT,’drop’,DBNAME)

Prerequisite:

Administrator account/password (root/’’)

Operations:

Expunge db instance

Needs interactive confirmationUnless variable ’force’ is defined in work area

Example:

>> dbROOT = tsdb(’J:mysql’,’root’,’’);>> tsadmin(db,’drop’,’bitstest’);


tsadmin(db,’trunc’)

Prerequisite:

User account/password (BITS/’’)

Operations:

Clean all db tables contents

Needs interactive confirmationUnless variable ’force’ is defined in work area

Example:

>> db = tsdb(’J:bitsdemo’,’BITS’);>> tsadmin(db,’trunc’);


Backup/Restore operations

Backup/restore operations save/load data in BACKUPDIRSave full time-series database, you can use fullbackup.

>> tsadmin(db,’fullbackup’,LABEL);It save a text file named where <date> is today date in’dd-Mmm-yyyy’ format.

Save all releases of a single series, use tsbackup.>> tsadmin(db,’tsbackup’,LABEL);

It save a matlab .mat file named <SERIENAME> <date> <LABEL>.matwhere <date> is today date in ’dd-Mmm-yyyy’ format.

Operations:Clean all db tables contentsNeeds interactive confirmation

Unless variable ’force’ is defined in work area

Example:

>> db = tsdb(’J:bitsdemo’,’BITS’);>> tsadmin(db,’trunc’);


tsadmin(db,’rm’, { series })

Prerequisite:

User account/password (BITS/’’)

Operations:

Deletes time-series and (part of) his history from db

Example:

>> db = tsdb(’J:bitsdemo’,’BITS’);>> tsadmin(db,’rm’, ’BDESPISDH’);>> tsadmin(db,’rm’, ’BDESPISDH’,

{datenum(’28/02/2007’)});


tsadmin(db,’cp/mv, { series }, db2)

Prerequisite:

User account/password (BITS/’’) on db1 and db2

Operations:

Copies or Moves a time-series and (part of) his history from one db toanother

Example:

>> db = tsdb(’J:bitsdemo’,’BITS’);>> db2 = tsdb(’J:bitsdemo2’,’BITS’);>> tsadmin(db,’cp’, ’BDESPISDH’, db2);>> tsadmin(db,’mv’, ’BDESPISDH’, db2);


Grazie dell’attenzione


workshop matlab bits database

Technology