BASE100 BASE 100, S.A.

www.base100.com

CARAVEL THE SAFE AND RELIABLE WAY

Caravel™ Reengineering

2 BASE 100, S.A.

© Copyright BASE 100, S.A. All rights reserved.

Information contained in this document is subject to changes without prior notice. These changes will be incorporated in

new editions of the document.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc., in the United States, other countries or both.

Every company name, product name, or service name may be trademarks or service marks of their respective owners.

Document: caravelreengineeringv5.9.2en

Version: 5.9.2

BASE 100, S.A.

http://www.base100.com

http://www.base100.com/

Caravel Reengineering 3

Contents

INTRODUCTION .................................................... 4 The Caravel™ Reengineering process main ideas .................. 4 The Caravel™ Reengineering objectives ................................. 5

Technical objectives ............................................................ 5

CARAVEL™ APPROACH .......................................... 6 Processes involved ................................................................. 6 Software Reengineering activities and Caravel™ Tools .......... 6 The abstraction level of Caravel™: reverse engineering

process ................................................................................... 7

CARAVEL™ REENGINEERING JOURNEY ................ 10 STEP 1: Insight - Looking into the unknown ......................... 10

A closer look to Caravel™ Insight functionality ................. 11 Insight in Automatic Conversion processes ...................... 12

STEP 2: Converting to 100% working Java EE ....................... 13 STEP 3: Data restructuring and migration ............................ 14 STEP 4: Understanding the Legacy System ........................... 15 STEP 4 II: Increasing abstraction level .................................. 16 STEP 5: Extending functionality ............................................ 17 STEP 0: Testing, verifying, certificating ................................ 18

Introduction

The BASE 100 team started the reengineering activity more than 15 years ago. Since then, millions of legacy

code lines have been modernized, focusing the activity in the IBM® midrange and

mainframe systems.

During this period, methodology, technology and tools, have been developed,

tested and put under continue evolution, conforming the Caravel™ moderniza-

tion suite. A clearly structured, highly efficient set, which covers from the first

assessment phases to the final Refactoring or Reengineering.

The BASE100 team has acquired a huge experience, after hundreds of successful

projects along the world. All the Caravel™ proposal, benefit from this experience.

Additionally, the Caravel™ suite includes the capability of automatic conversion

from RPG or COBOL Legacy Systems, to Java. This feature can be used as a final objective or as an intermediate

valuable step in the Reengineering process.

Caravel™ methodology takes account of the inherent risks of any reengineering project, offering the appropri-

ate answer at every stage of the process. Additionally, Caravel™ relies in the use of advanced tools, to reduce

the manual effort, automatizing all the critical processes.

Both lines reduce the time and costs, minimizing the risks. As result, Caravel™

offers a safe and reliable Refactoring and Reengineering wayi.

The Caravel™ Reengineering process main ideas

Caravel™ Reengineering allows Re-structuring and Re-writing a Legacy System

preserving its core functionality. Moreover, Caravel™ offers a complete life cycle

for the whole process, with special attention in prevent and eliminate the risks of the process. The base lines in

Caravel™ Reengineering are:

1. Detailed Risk analysis.

All the project planning comes after a detailed analysis of the inherent risks of the reengineering process. Prior

to start the project, every risk is evaluated, and then during the process, when needed, there is always ready

the appropriate answer.

2. Keeping the whole process under control.

In Reengineering projects, Timeline, milestones and

budget, are difficult to keep under control. But by means

of Caravel™ all of them are measurable and predictable.

Any deviation is promptly detected.

3. Frequent Deliverables.

Deliverables are the best way to verify the real situation

of the process. Every milestone has associated a deliver-

able that adds value to the legacy asset and acts as

checking point. From the first step of the process, customer can verify the real progress.

Reengineering is the examina-

tion, analysis and alteration of

an existing software system to

reconstitute it in a new form,

and the subsequent imple-

mentation of the new form.

The main goals are increase

reliability and maintainability.

Caravel™ methodology is

based in a detailed analysis of

the inherent risks of any

reengineering project. At eve-

ry stage of the project the ap-

propriate answer, is always

ready.

Caravel Reengineering 5

The Caravel™ Reengineering objectives

The Reengineering process must be a cost effective solution for the system owner. This is the first responsibil-

ity in every Caravel™ modernization project. Our technical team has the training, the experience and the men-

tality to achieve this goal.

A cost effective solution must be a combination of benefits, reaching all the technical objectives, while main-

taining the process under a strict costs and time control.

Technical objectives The technical objectives can be reached at different moments of the modernization project. According to char-

acteristics and priorities, these objectives will be accordingly organized in the project timeline. (Therefore, the

list below is not in sequential order.)

1. Improving maintenance and reliability.

Powerful reverse engineering tools, allow accurate analysis and assist the way for effective Componentization

and code Re-writing. The detailed object analysis supports an effective code cleaning processes.

2. Maintaining and extending its functionality.

Identical functionality is fully guaranteed, either when it comes from pro-

grams or from the operating system facilities. Highly productive Direct

Reengineering tools allow extending functionality easily at any stage.

3. Migrating data to a modern structure.

All data is analyzed, restructured and migrated. Then, by means of auto-

matic tools, the data consistency is completely tested.

4. Moving the system to a new platform.

From the early stages of the Reengineering process, a fully working converted system can be run under any

standard platform, maintaining functionality and performance. This converted system allows the legacy plat-

form to be turn off. Moreover, the converted system offers a powerful way reverse reengineering activities.

5. Tests, verification and certification.

In every stage of the process, tests are performed to verify both functionality and performance. As the consec-

utive test sets are successfully passed, the system goes toward the complete certification.

6. System Documentation.

The reengineering process includes the building of a new system technical and functional documentation.

The new technical documentation is generated after a static analysis by means of Caravel™ Insight tool. All the

components, characteristics and relationships are deeply studied and documented. Every object is analyzed

and all the functionality encapsulated in it, is extracted and stored in a semantic tree structure. After this, are

built all the properties descriptions, flow diagrams and related documentation of the single object.

All the functional documentation is built by means of Caravel™ Sequencer tool. Performing automatically a set

of use-cases, the real processes inside the running system are extracted. These bring all the necessary infor-

mation about the internal behavior of the system.

Both technical and functional documentation constitute a complete description of the system.

Caravel™ methodology is ori-

ented to achieving all the tech-

nical objectives, while main-

taining the process under a

strict costs and timing control.

6 BASE 100, S.A.

Caravel™ approach

The Refactoring or Reengineering journey includes several steps; in which different processes are involved. In

every step specialized tools are used for specific activities.

The aim of the Reengineering is (1) to understand the Legacy System and (2) document and extract the infor-

mation contained in it to reach the necessary level of abstraction, then (3) rewrite, and (4) extend functionality.

Processes involved

Caravel™ Reengineering encompasses a group of processes:

Analysis. Reverse engineering. Conversion. Data migration. Re-documentation. Re-structuring. Forward engineering.

Software Reengineering activities and Caravel™ Tools

Activities Goals Tools

Static Inventory and Analysis. Complete documentation. Reverse reen-

gineering, inventory.

Code conversion to Java EE. To build a completely working system,

maintaining the same functionality.

Data restructuring and migration. Data consistency verified. Most efficient

data structure.

Test, verification and certification. Continuous control of the results, at every

stage of the process.

Functional Dynamic Analysis.

Sequence diagrams. Reverse engineering

for Componentization. Use-Case man-

agement and modeling. Business rules

extraction.

Forward Engineering. Extending functionality.

Caravel Reengineering 7

The abstraction level of Caravel™: reverse engineering process

Caravel™ Process and tools are high effective to extract logic and design information from source code. To fully

accomplish the objectives of the Reengineering process a complete extraction of Legacy System information

and an adequate level of abstraction are mandatory.

The build of the level of abstraction is a progressive process, in which stages are reached step by step.

1. Program and data structure information and relationships of all them. 2. Object models. 3. Data and/or control flow models. 4. Deriving procedural design representations. 5. Entity relationship models.

As the abstraction level increases, technicians are provided with information that will allow easier understand-

ing of the system.

The Caravel™ set of tools helps in the different stages of the process, providing a guide, to optimize and reduce

the manual effort.

Caravel Reengineering Journey

10 BASE 100, S.A.

Caravel™ Reengineering Journey

STEP 1: Insight - Looking into the unknown

The first step performs a deep analysis of all the Legacy System, creating an inventory of all the objects and its

relationships and looking inside the structure and the logic of every object. For that purpose Caravel™ Insight

analyses the system, retrieves every object, extracts all the information and stores it in a semantic tree. As re-

sult, is created a data base with the Object Repository containing all the information. This information is used

by the Caravel™ tools to achieve different processes and constitute a complete documentation of the system.

Types of objects, relationships,

anomalies, logic included, complexi-

ty analysis, flow charts, are provid-

ed, to fully support the technical

tasks.

When the aim is the automatic code

conversion, this analysis provides a

complete check list of all the poten-

tial issues present in the Legacy Sys-

tem.

1. Complete inventory of the system.

2. Identify and solve anomalies: duplicated sources, unused objects, unreachable code, etc.

3. Generate the complete database with the objects attributes, their relationships and the

logic included.

4. Generation of the Data model matching the AS/400 naming with Java EE names (table

names, column names, programs, subroutines, etc).

5. Creation of the entity relation between the different tables.

6. Create business documentation for the objects that will be used on the reengineering

process.

Caravel Reengineering 11

A closer look to Caravel™ Insight functionality Caravel™ Insight possesses the capacity to deeply analyse the original system. His functionality is configurable,

oriented to adapt the information management to the specific customer needs.

All the system is showed in a completely interactive and navigable way, allowing the user to a flexible and

highly productive usage and a fast, low effort understanding of the Legacy System structure and organization.

Among other features Insight includes:

Extraction from AS/400. Insight connects to the AS/400 and extracts all the objects of the Legacy System, building a data base in an external open platform.

System’s Documentation: Automatic and complete documentation of the entire system, with configurable levels of detail. These reports can also be generated in HTML format or exported to CSV from specific ta-bles.

Consulting for the modernization: Evaluation according to standards of the complexity and efficiency of each component. The data utilize for the evaluation are exportable in CSV format and the user may re-evaluate or personalize them in order to create his own standards.

Refactoring/Reengineering: With the analysis performed and the information generated it is possible to obtain a better understanding about those improvable aspects and pre-conversion changes, in order to ob-tain a more successful converion. On the other hand, the information provided by Caravel Modernization Insight allows to simplify the manual re-writing of a system into another language, as well as its refactoring in RPG.

Management and analysis of components: Analysis of all the components of the Legacy System (libraries, programs and files), and their relationship, external dependencies and internal characteristics.

Insight looks inside the object logic and builds the related Semantic Tree (see related document) that will be later used by other analysis processes.

12 BASE 100, S.A.

At this stage all the diagram-flow are built, according a flexible and well organized template, including many features as optional pseudo code generation.

The functionalities provided by Caravel Modernization Insight offer a complete report to determine the ac-tions to carry out in order to obtain a successful Reengineering.

Insight in Automatic Conversion processes Caravel™ Insight not only possesses the capacity to analyse the original system. In case of automatic conversion

to Java, includes the necessary functionality for the upload and management of the system to convert.

Additionally has the capability to analyse and prepare the post-conversion to Java process, by utilizing both

automatic and manual procedures.

Caravel Reengineering 13

STEP 2: Converting to 100% working Java EE

Caravel™ Converter translates the whole functionality of the original system to Java EE.

This system uses the same set of data

that the original Legacy System. This al-

lows performing a complete set of tests,

verifying code translation and certificat-

ing the same functionality.

As result the converted system is totally

working maintaining 100% the same

functionality. Now the legacy platform

could be switched off and the converted

system can be run under any open envi-

ronment.

The converted system is the basic mile-

stone for the following steps in the reen-

gineering process and brings an im-

portant first level of componentization.

The Converted System is 100% pure Java

EE and can be modified using any stand-

ard Java IDE.

1. System converted to Java, 100% working and installa-

ble in any open platform.

2. 100% identical functionality in both Java and original

system.

3. New role for the RPG programmers as business logic

analysts.

4. Eliminates pressure with the project timeline: the sys-

tem is working in open environment.

5. Integration with external applica-tions6. Create busi-

ness documentation for the objects that will be used on

the reengineering process.

14 BASE 100, S.A.

STEP 3: Data restructuring and migration

Caravel™ Data Analyzer migrate data to a new RDBMS structure.

Data restructuring and migration, is

performed in a stage after the Legacy

System conversion, due to several

reasons:

This eliminates any interference from

data modification during the conver-

sion process. Functionality can com-

pletely be tested and certificated.

Additionally, in scenarios with several

Legacy Systems, if any of them con-

tinue operating and sharing the same

set of data, there is not conflict be-

tween both (original and converted).

In fact the converted system can

share with other Legacy Systems the

same set of data, without limitations.

In this STEP 3, by means of Data Analyzer, Caravel™ Reengineering performs:

1. Analysis of the legacy data structures (LF and PF) and transformation to a Relational DB Model. This model

can be personalized, modifying names and types of data structures:

a. Changing names or labels on tables or col-

umns

b. Modifying SQL column data types

c. Adding Foreign keys or indexes

d. Adding new columns to extend functionality

2. Extraction of the legacy data in raw format, insertion in the new Relational DB Model.

3. Analysis of the quality of the RAW data. Data Analyzer verifies any error in data such as wrong numeric or

data values. According with the specific project needs different verifications can be performed.

4. Population of the new Relational DB.

1. SQL Data model creation accessible by BI

applications, Excel, etc.

2. Data cleaning and data integrity checking.

Caravel Reengineering 15

STEP 4: Understanding the Legacy System

Caravel™ Sequencer performs a dynamic analysis of the converted system: extracts the Execution Sequences,

the Call Trees of different components and all the Flow Diagrams.

Caravel Sequencer performs an analysis programs during execution, allowing the identification of eligible com-

ponents. This will direct the further reengineering steps.

After the Test Cases defined with Caravel™ Test Maker, Sequencer builds a complete set of logs and then deep-

ly analyses all of them.

16 BASE 100, S.A.

STEP 4 II: Increasing abstraction level

The Caravel™ reengineering offers increasing levels of abstraction.

The components are characterized as business logic and data.

The presentation layer may use different technologies.

REST services oriented architecture can be also used.

Presentation layer

Static and dynamic information can be extract to from legacy

screens to generate new personalized screens.

Information about the screens and their use by the Legacy System is

obtained using TestMaker. A detailed log (in JSON or XML format)

are created for every screen execution.

Data Access Components: DAOs

Are generated after the conversion phase by means of the Caravel™

Converter, and then enhanced using the information extracted from

the system.

Later, using the JSON or XML specification, Caravel™ Expand generates the new screen prototype. This mecha-

nism is similar to the automatic screen Expand generation based on the database model structure.

1. New analysis documenta-

tion (E/R model, flow an se-

quence diagrams, etc).

2. Scalable and maintainable

system organized in business

components.

Caravel Reengineering 17

STEP 5: Extending functionality

Caravel™ Expand, allow building new functionality in a highly productive way.

New programs can directly con-

nect with the data structure and

components of the Converted Sys-

tem.

After the reengineering project, in

the new system, reengineered

components will coexist with ut-

terly new components build with

Caravel™ Expand or build by

means of any other development

platform.

1. Converted application based in the standard frameworks used in the

company.

2. Progressive migration of the business components: as the system is

working and certificated in Java, the customer can decide migrate only

some parts of the application.

3. Programs accessible by REST / SOA.

18 BASE 100, S.A.

STEP 0: Testing, verifying, certificating

In case of automatic Conversion, the certification process allows to the users to switch safely to the new sys-

tem.

Caravel™ Test Maker prepares and executes tests than compare the behavior of both systems at every stage of

the reengineering process.

Test Maker includes

all the functionality to

plan, create and per-

form group of tests

that verify all the

functional aspects of

the Legacy System.

Caravel Reengineering 19

i Refactoring and Reengineering.

In this document we focus in Reengineering, while considering Refactoring a more simple process. From the Caravel™ point of view, the Refactoring process is more reduced due that intends to preserve the actual behavior of the system.

According to Martin Fowler definition: “Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth do-ing". However the cumulative effect of each of these transformations is quite significant. By doing them in small steps you reduce the risk of introducing errors. You also avoid having the system broken while you are carrying out the restructuring - which allows you to gradually refactor a system over an extended period of time.”

A typical approach of Refactoring is automatic code conversion; which is performed by Caravel™ Converter tool.

Even we found a lack of precise definition of Software Reengineering; we consider that the aim of this process is different and more ambitious when compared with Refactoring. According our understanding, Reengineering intends to build a new system that covers the original functionality but expands it and often includes new capabilities. This new functionality is implemented not following the design, implementation or technological lines of the original system, but with a new ap-proach.

From a practical point of view Reengineering is a process that requires 2 to 10 times more effort.