GEOSS components testing procedure

14th ADC – Pretoria – 20th May 2010

Outline

• GCI components selection– Selection criteria

• Tests Roadmap– 1st Phase: Clearinghouse selection– 2nd Phase: GeowebPortal selection

• Results• Conclusions

GCI-CT: Background

• 2009 GEO VI Plenary endorsed the GCI architecture recommended by the IOC TF.– single GEO Web Portal (GWP) and a single

Clearinghouse (CH)

• GCI Coordination Team (GCI-CT) created to define the selection process and oversee the long-term operations of GCI.– GCI-CT Kick-off Meeting, Feb 23-24, 2010.

GCI Contributors

• Component and Service registry (CSR): USGS/George Mason University (USA, Dept. of Interior)

• Standard and Interoperability register : IEEE

• Best Practices Wiki: IEEE

• GEOSS Clearinghouses (CLHs, 3): • Compusult (Private company) • ESRI (Private Company)• USGS (US)

• GEO Web Portals (GWPs, 3): • Compusult• ESRI• ESA/FAO (GEO Partic. Org.)

Operational scenario: single versus multiple GCI components

GWP1 GWP2 GWP3

CSR

CL1 CL2 CL3

USER

Current GCI configuration IOC phase 2008-2009

(1st Phase)Single CL /

multiple Web Portal GCI

configuration operational phase

2010-2015

(2nd Phase)Single CL / single Web Portal GCI configuration

operational phase 2010-2015

GWP1 GWP2 GWP3

CSR

CL

USER

GWP

CSR

CL

USER

Selection Criteria

Criteria Description

1 “The GCI component provider commits to providing the necessary level of resources, (financial – both "in-kind" and actual, personnel, etc.), that will enable the sustained operation, enhancement and maintenance of the GCI during the period 2010 – 2015”

2 The GCI component provider identifies the GEO Member or Participating Organisation "sponsoring" their operational engagement with GEO;

3 The GCI component provider agrees to accept, abide by and implement the

requirements set out in the GCI Consolidated Requirements document, dated 2 March 2010;

4 The GCI component provider gives a guarantee that should it be obliged to cease its GCI support activities, a 3rd Party can fully take over the operation and maintenance of the corresponding GCI component(s), free from any restraints arising from software copyrights, IPR and with the availability of all necessary technical documentation;

5 A GCI capabilities evaluation is undertaken to verify the compliance of the GCI component with the Consolidated Requirements and a rating is provided for objective comparison of offerings where multiple offers are made;

6 A general usability assessment is conducted and offerings of GCI components are rated based on their subjective suitability for end-user needs.

Compliance to Consolidated Req.

• Actors:– GCI Coordination Team (CT )– Components providers– Test team

• Joint Research Center (JRC, EU )• National Institute for Space Research (INPE, Brazil)

• GCI Consolidated requirements (Common/Specific)– Service availability,– Quality management of content,– General user functionality– ...

GCI Component Testing Roadmap – Phase 1

•Responses to GEO SEC letter with criteria (19 Mar):Action: Providers

•Self-Testing by CSR, CL, GWP (22 to 26 Mar)Action: Providers

•Phase 1: CSR, CL testing (29 March to 13 Apr)Action: JRC & INPE

•Report of CSR, CL testing (20 Apr)Action: JRC & INPE

•Clearinghouse to be selected for test phase 2 (27 April)Action: GCI CT

GCI Component Testing Roadmap – Phase 2

•GWP to reconfigure their systems (28 April to 6 May)Action: GWP Providers

•Phase 2: GWP testing with selected CL (7 to 17 May)Action: JRC & INPE

•Phase 2: GWP Usability testing with sel. CL (17 to 20 May)Action: US EPA

•Interim Reports of 1) GWP testing and 2) Usability testing (28 May) Action: 1) JRC and INPE, 2) US EPA

•Final Reports of 1) GWP testing and 2) Usability testing(Mid-June) Action: 1) JRC and INPE, 2) US EPA

1st Phase: CSR & Clearinghouse Selection

Source: “AIP-2 Design Review Catalogue, Clearinghouse, Registry, Metadata (CCRM) WG Use Case Review”, Josh Lieberman – GEOSS AIP-2 Design Review, 2 December 2008

1st Phase: tests (CSR)

Code Test NameJRC/INPE

Provider Movie Doc. Time

CSR-1 Creation of user's login on CSR Both No No No29/03/2010

(11:12)

CSR-2Component registration and modification JRC No No No

29/03/2010 (14:17)

CSR-3Service Registration and modification JRC No No No

30/03/2010 (14:16)

CSR-4CSR Management by administrator INPE Yes Yes No

13/04/2010 (13:00 GMT)

CSR-5CSR content visualization and simple search INPE Yes No No

13/04/2010 (13:00 GMT)

CSR-6CSR content visualization and complex search JRC No No No

14/04/2010(9:12)

CSR-8 (opt)

Login maybe shared with other GCI registries INPE No No No 05/04/2010

CSR-11 Data protection INPE Yes Yes Yes13/04/2010

(13:00 GMT)

1st Phase: tests (Clearinghouses)

Code Test NameJRC/INPE Provider Movie Doc Time

CL-1ClearingHouse query to CSR

JRC Yes No No

31/03/2010 (9-10: JRC) (WebEx:14-16 Compusult)

01/04/2010 (WebEx 14:16 ESRI)

06/04/2010 (WebEx 14:16 USGS)

CL-2GEOSS catalogs harvesting by CL

INPE Yes No Yes

06/04/2010 Compusult

08/04/2010 USGS

Wednesday 14/04/2010 at 15:00 (GMT) ESRI

CL-3Query GEOSS registered resources via CL

JRC No No No

01/04/2010 (9:12), Compusult, Rescheduled: 13/04/2010

01/04/2010 (9:12) ESRI

01/04/2010 (9:12) USGS

CL-4ClearingHouse administration

INPE Yes Yes No

06/04/2010 Compusult

08/04/2010 USGS

Wednesday 14/04/2010 at 15:00 (GMT) ESRI

CL-6 Data protection JRC Yes Yes Yes

07/04/2010 (WebEx 16:18 Compusult)

01/04/2010 (WebEx 16:18 ESRI)Rescheduled: Wed. 14/04/2010 at 15:00 (GMT)

06/04/2010 (WebEx 16:18 USGS)

1st Phase: tests (results)

CSR-1 CSR-2 CSR-3 CSR-4 CSR-5 CSR-6 CSR-8 CSR-11

Provider Pass Pass Pass Pass Pass Pass Pass Pass

CL-1 CL-2 CL-3 CL-4 CL-6

Provider1 Fail Pass Fail Pass Pass

Provider2 Fail Pass Fail Pass Pass

Provider3 Fail Pass Pass Pass Pass

• Component and Services Registry (CSR)

• Clearinghouses

CSR test: exampleTest Description

Test Title Service registration and modification

GCI Test Identifier CSR-3

Overall description and test objectives

Upon the registration of a Component, the publication user can enter, edit, and save registration information on one or more Service interfaces. Publication user uses any popular Web browser for interaction with the CSR [R52] . Publication user can select and associate with a service an existing standard or special arrangement from the Standards and Interoperability Registry (SIR) [R8], or if unavailable, is able to nominate a new standard or arrangement directly into the SIR process [R30]. In addition the user shall be forbidden to modify or delete any service information registered by another user.[ID: R8, R30, R52]

Test Using Firefox, Safari, Chrome or Internet Explorer (other than the brand used in CSR-1 or CSR-2), user:1.logs into the CSR with previously set credentials.2.after repeating CSR-2 step # 3 to reinsert a new Component, the Tester creates a new Service entry associated with a Component entry that they previously contributed. Service entry describes a Catalog Service.3.updates the Service entry with a link to the ISO23950 catalog service standard and the ISO19115 metadata standard, described in the SIR.4.begins, but does not save, the nomination of a new “special arrangement” using forms that would post a new entry into the SIR.5.saves the Service record in the CSR.6.tries without success to repeat to update and then delete a Service record entered by someone else.7.logs out of the CSR.

List of GCI or non-GCI elements

GEOSS Component and Service Registry (CSR)

Test Environment (HW and SW): Hardware and software used during the test

GEOSS Component and Service Registry (CSR), <version TBC>Web client browser (Firefox, Safari, Chrome or Internet Explorer) different from the one used in CSR-1 and CSR-2

List of interfaces being verified during the test

Native registration interface, HTTP

Tests pre-requisites Tests CSR-1 and CSR-2 executed successfully

Pass/fail criteria Pass: Publication user successfully logs into the CSR, creates, updates, and saves a Service Record associated with an existing Component record and two standards/special arrangements using a browser different than used in CSR-1 or 2. In addition, the user cannot alter the Service Record entered by another user.Pass with minor problems or observations: identical as for the “pass” criteria. The web interface might lead to remarks if it is confusing, not user friendly, etc …Fail: Registration interface fails to save or update a Service record, browser is not compatible, or links to standards are not made or preserved. User succeeds to modify or delete the Service record registered by another user.

CSR test result: example (1)

Date 30th March, 2010

Tester Lorenzino Vaccari (JRC)

Detailed Results

Test1 The tester logs into the CSR with previously set credentials.

Execution The test was executed by using the Chrome browser.

Pass/Fail Pass

Test2 Creating of a new Service entry associated to a newly created Component.

Execution By using the Chrome browser, the tester created a new Service entry associated to a newly created Component.

Pass/Fail Pass

Test3 The tester updates the Service entry with links to the ISO23950 catalogue service standard and ISO19115 metadata standard, described in the SIR.

Execution By using the Chrome browser tester updated the Service entry with links to the ISO23950 catalogue service standard and ISO19115 metadata standard. The tester associated the Service to the following SIR entries (only one choice is possible): OGC Catalogue Service 2.0.2 (Z39.50 Binding), OGC Catalogue Service 2.0.2 – (ISO Profile 1.0.0, HTTP Binding)And to the following Supportive Information Standards or Special Arrangements: ISO 19115:2003 Geographic information – Metadata

Pass/Fail Pass

Test4 The tester begins, but does not save, the nomination of a new “special arrangement” using forms that would post a new entry into the SIR.

Execution By using the Chrome browser, the tester began and did not save the nomination of a new “special arrangements” using forms that would post a new entry into the SIR.

Pass/Fail Pass

CSR test result: example (2)

Test5 The tester saves the Service record in the CSR.

Execution By using the Chrome browser, the tester saved the service record created in previous tests.

Pass/Fail Pass

Test6 The tester tries without success to repeat to update and then delete a Service record entered by someone else.

Execution By using the Chrome browser, the tester tried without success to repeat to update and then delete a Service record entered by someone else.

Pass/Fail Pass

Comments The tester identified the following bug: the tester tried to delete the service owned by another user by substituting the ID number into the http address. After that, the system deleted the service.

Test7 Logging out of the CSR

Execution By using the Chrome browser, the tester logged out of the CSR portal.

Pass/Fail Pass

Final Results

Comments The tester tried to delete the service by substituting the ID number in the http address for a non proprietary record and then the system deleted the service.

Pass/Fail Pass with minor problems or observations

Test multiplication…CL-1

ESRI Compusult USGS

CSW 2.0.2 Z39.5 CSW 2.0.2 Z39.5 CSW 2.0.2 Z39.5

2.a 2.b 2.c 2.d 2.a 2.b 2.c 2.d 2.a 2.b 2.c 2.d 2.a 2.b 2.c 2.d 2.a 2.b 2.c 2.d 2.a 2.b 2.c 2.d

GCI Component Testing Roadmap – Phase 2

•GWP to reconfigure their systems (28 April to 6 May)Action: GWP Providers •Phase 2: GWP testing with selected CL (7 to 17 May)Action: JRC & INPE •Phase 2: GWP Usability testing with sel. CL (17 to 20 May)

Action: US EPA•Interim Reports of 1) GWP testing and 2) Usability testing (28 May) Action: 1) JRC and INPE, 2) US EPA•Final Reports of 1) GWP testing and 2) Usability testing(Mid-June) Action: 1) JRC and INPE, 2) US EPA

2nd Phase: tests (GWP)

Code Test Name JRC/INPE Provider Movie Documentation

GWP-1 Simple search & browse resources INPE No No No

GWP-2 Complex search and browse resources JRC No No No

GWP-3Complex search and browse resources using a globe INPE No No No

GWP-4 Easy search and quick browse JRC No No No

GWP-5 Help to user and User feedback Both No No No

GWP-6Capability support multiple concurrent access JRC No No No

GWP-7 Quality of GWP content INPE Yes Yes No

GWP-8 Browse GEO-RSS JRC No No No

GWP-9 Redeployment of GWP application INPE Yes Yes Yes

GWP-17 Data protection INPE Yes Yes Yes

GCI-CT: recent Activities

• Weekly telecons since GEO Plenary

• Draft functional test procedures (Feb 18, 2010)

• GCI-CT Kick-off meeting (Feb 23-24, 2010)– Consolidation of Work Plan & Consolidated

Requirements– Definition of Selection Process and Test Logic– Agreement of Weights on criteria– Draft Letter with criteria to GCI Providers

• Letter sent to GCI Component Providers(2 March 2010)

GCI-CT: recent Activities

• Meeting GCI-CT and Test Teams (INPE, JRC)(3 March 2010)– Optimization of test logic– Consolidation of Work Plan– Establishment of rating system

• Assessment of Test Procedures by Test Teams (INPE, JRC) (8 March 2010)

GCI-CT: Deliverables to ExCom-18

• GCI-CT Terms of Reference: ExCom Document 8

• Recommendations for Long-Term Operations of the GCI: updated version of 5 March 2010

• Action (Work) Plan: updated version of 5 March 2010– Note, further update still needed to reflect the Test

Phases and associated milestones dates.

• Consolidated Requirements – updated– Note, further update still needed to include a new

requirement relative to the time notice (90 days) for a GCI Provider to inform the GEO Secretariat in case of their cessation of GCI operations.

Conclusions

• The process is underway that will lead to a recommendation on the single Clearinghouse and GEO Web Portal being prepared for EXCOM, with the expectation that this recommendation can be made available by mid-June 2010.

• It is important that this timescale is adhered to, as there is a clear need to complete the selection process as soon as possible, in order to allow the selected GCI Component Providers to work together to ensure that a truly operational GCI is implemented in advance of the GEO Ministerial Summit.