csts-file transfer service discussions (2)

22
19/05/2011 CSTS File transfer service discussions CSTS-File Transfer service discussions (2) CNES position

Upload: tomai

Post on 15-Jan-2016

48 views

Category:

Documents


1 download

DESCRIPTION

CSTS-File Transfer service discussions (2). CNES position. File Content Description Define manifest files identifying File content (to the level that the recipient needs to know) File format? Processing Instructions Combine with content description? Pull Model, Push Model, or both? - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS-File Transfer service discussions (2)

CNES position

Page 2: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

• File Content Description– Define manifest files identifying

– File content (to the level that the recipient needs to know)– File format?

• Processing Instructions– Combine with content description?

• Pull Model, Push Model, or both?• [Mouy Francois-Xavier] No needs identified yet for push model (but still a little bit of work to do)

• Subscription and automatic push? [Mouy Francois-Xavier] for push model • Subscription and notification of new data? [Mouy Francois-Xavier] how send notifications if

provider not bind. Bind return could notify a list of avaliable data

• File Management Services (see dedicated slide)• File Directory / Catalogue Servcies• File Management [Mouy Francois-Xavier] for push model

– Create / Delete– Move– Local copy

• Directory Management [Mouy Francois-Xavier] for push model – Create / Delete / Copy

• Storage Control [Mouy Francois-Xavier] needs to be configurable – Duration of Storage– Maximum Volume (per client?)

Page 3: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

• Security (see dedicated slide)• Off the shelf FT protocols support security but must be configured

and depending on features used may need supporting infrastructure• Firewall issues• Access Control

– At what level? (mission, individual accounts, …) [Mouy Francois-Xavier] Mission level seem to be the right level

– Access Control Groups – More complex schemes

• What level of privacy / access control is needed? [Mouy Francois-Xavier] We need to be able to provide this kind of features before security person's asks – E.g. can user X see that data are stored for user Y? [Mouy Francois-

Xavier] he shouldn't ;-) • Key management issues• Others?

Page 4: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Reading Martin initial thoughts.– Protocol choices are wide. If we use one or other

protocol, it comes with all this complexity (advantages, drawbacks).

– Some features of standards COTS are not useful in our context, but you have to tune and secure those features anyway.

– Most of the difficulty come with the push model (security, behavior), the pull model is very near the existing offline model.

Initial CNES position:– Firstly, we thought we only need the pull model which

is quite simple than the push model. – Furthermore, File transfer function is already done by

other ways. (ftp servers in parallel)– Therefore, why redo it ?

Page 5: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

• However, Experience show us that deploying a file transfer feature with common COTS is not quite easy that it is on paper (e.g. with ftp: passive form, active , dynamic port ranges, firewalls, user management, … ).

• Using CSTS could simplify the integration because the network security aspects are already done. A part of the reporting too (notifications).

• CNES think CSTS could be a good « Secured Pipe » to the Ground center.– A binded instance is maintained (Heartbeat)– The Start/Stop mechanisms could be useful to drive

providers behavior– We are able to use CSTS services via a high level of

firewalls system

Page 6: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Candidate standards (from initial thoughts)

• File Transfer Standards– FTP, FTPS, SFTP (SSH based FT)– HTTP / HTTPS ?– Others?

• CCSDS Standards– XFDU ?– Others?

Page 7: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

XFDU (xml formatted data unit)

• GOAL : organizing transferred data in an Open Archive environment (OAIS)

A container (zip file) with

a descripition of its content in a manifest file.

The logical description points to the physical content which can be either local or distant

Page 8: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

XFDU description

• The Information Package Map contains the logical view of the package. It’s a hierarchical xml tree representing the content of the package. Each leaf of the tree is a Content Unit and can be referred to from the other parts of the package (i.e. information is linked by the Content Unit reference ID).

• The Data Object Section contains all the physical information needed to get the file objects out.

• The Metadata Section records all of the metadata for all items in the package.

• The Behavior Section associates executable code with the content of the package.

Page 9: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

XFDU manifest structure

The structure map is a hierarchical view of logical references which are pointing to physical data, metadata and specific behaviors.

Page 10: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

XFDU : benefits

• Unique Identification of the data ( ContentUnit ID)

• Interoperability by using an XML schema description

• Hierarchical structure of the information• Data physical access (DataObject) separated

from the logical concepts (ContentUnit)• Metadata associated with the ContentUnit• Specific processes associated with the

ContentUnit (BehaviorObject)

Page 11: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

XFDU : drawbacks

XFDU developed for approaching needs but not totally the sames (multi-parts, files, cdrom)

• Few implementations due to standard complexity.:– NASA XFDU library, ESA SAFE’s archive for

ENVISAT data, CNES’s PAIMAS

• Behavior function not really validated

Page 12: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS-XFDU : conclusion

• The data organization has been studied earlier by the MOIMS/IPR group. XFDU is an answer to csts concerns with the following conditions: – Needs of an XFDU « light » to answer the

requirements.– Needs to study the behavior statement.

.

Page 13: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS-XFDU : conclusion

• XFDU (simplified one) Could be a good system to integrate the file transfer feature.

• The behavior side of XFDU could be a response to File transfer and more …

• CNES thinks it could be a possible cooperation and some people are interested to study those aspects …

Page 14: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Page 15: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Some use cases at CNES

• Distant management of stations– Management by xfdu of TM archived in station– TM playback via CSTS-offline service

• Inter-facilities scheduler– Distant scripts used by the scheduler managed by

xfdu– Launch via services CSTS– Re-use of CNES well-known scheduler GUI

• …

Page 16: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Let’s take an example….

XFDU over CSTS for playing recorded telemetry

Page 17: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

Telemetry transfert and play

• A facility wants to transfert simulated telemetry to a station and to be able to play it back later.

• The user build an xfdu package (X1) with the TM file and two scripts (play and rewind)

• All the configuration and security concerns to be solved by the management layer.

Page 18: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS FT

user

TM 1

PLAY

REW

XFDU X1

CSTS FT

provider

bind

STEP 1

CONNEXION

Page 19: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS FT

user

TM 1

PLAY

REW

XFDU X1

TM 1

PLAY

REW

XFDU X1

CSTS FT

provider

bind

START PUT X1

STEP 2

XFDU TRANSFERT

Page 20: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS FT

user

TM 1

PLAY

REW

XFDU X1

TM 1

PLAY

REW

XFDU X1

CSTS FT

provider

TM 1

PLAY

REW

bind

START PUT X1

INSTALL

STEP 3

XFDU AUTOMATICINSTALL

Page 21: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS FT

user

TM 1

PLAY

REW

XFDU X1

TM 1

PLAY

REW

XFDU X1

CSTS FT

provider

TM 1

PLAY

REW

bind

START PUT X1

START LIST

START LIST X1

PLAY, REW

X1

INSTALL

STEP 4

The user lists all available xfdus

and their behaviors

Page 22: CSTS-File Transfer service discussions (2)

19/05/2011 CSTS File transfer service discussions

CSTS FT

user

TM 1

PLAY

REW

XFDU X1

TM 1

PLAY

REW

XFDU X1

CSTS FT

provider

TM 1

PLAY

REW

bind

START PUT X1

START LIST

START LIST X1

START RUN X1 PLAY

TM 1

PLAY, REW

X1

INSTALL

STEP 5

The user plays the TM1 telemetry back