computational science & engineering department cse 1 software engineering tools – can they be...

1

Computational Science & Engineering Department

CSE

Software Engineering Tools –

can they be of any practical use?

Dr Chris Greenough

Software Engineering GroupComputational Science & Engineering

Rutherford Appleton Laboratory

[email protected]

2


CSEElements of this presentation

• Motivation and background– The CCP Review– The EPSRC SLA Review – The CCP scoping study

• SESP - software engineering for CSE!– Overview of programme– Interactions within CSE

• A starting point – legacy software and immediate requirements

• A transformation process – guided tour

• Access to resources– Software tools server– Web site (www.sesp.cse.clrc.ac.uk)– QA Portal (www.qaportal.cse.clrc.ac.uk)

• Priorities for the CSE science programme

3


CSEMotivation - a CSE perspective

"To ensure that UK researchers benefit from the best computational methods, supporting them through collaboration, software development, facilities and education” (CSE Mission statement).

What does this software development require?

– doing excellent science and engineering (often in collaboration)

– devising or enhancing complex computational models

– developing advanced & efficient numerical techniques

– designing, implement & supporting quality software

What does quality software mean to CSE (or what should it mean)?

4


CSEThe CCP Review

CCP Review - Recommendations 9 & 10

– CCPs should expand their use of modern software development tools, in collaboration with computer scientists and software developers in academia and in industry.

– CCPs should respond to the challenge of "GRID-enabling" existing applications and designing new projects with the GRID in mind from the outset.

5


CSEThe EPSRC SLA Review

SLA Review - Recommendations 3 & 12

– CSED should develop mechanisms to disseminate best practice across all of the CCPs

– CSED and EPSRC should work together to encourage greater involvement of the Computer Science and Mathematics academic communities.

6


CSEThe CCP scoping study recommendations

• Develop a common and extensible framework and a technology set for CCP GUIs and data visualisation.

• Use component architectures and other mechanisms for increasing code interoperability and for managing and transitioning legacy codes.

• Establishing a common SourceForge-like resource,

• Refresh the software skills and tools employed by the computational science community including the standards of software development.

• Accelerate the use of performance tool in scientific applications on parallel computers and parallel programming tools.

• Define the CCP requirements for problem-solving environments and work-flow management in GRID applications.

7


CSECCP Scoping Study - Summary

• The recommendations have as common factors the adoption of

new technologies and interfacing with products of the latest

computer science research.

• In order to retain a competitive advantage, it is crucial to reduce

the lag between development in such research and associated

test beds, and exploitation in actual applications.

8


CSEThoughts from Huub van Dam (apologies - taken from an Email!)

There should be plenty of issues to discuss. Just to start with I include a list of things that bother me:

Software tools:

- Which development environments to use?

- Which Code analysis and verifications tools?

- Code version tools? We currently use CVS however this

does have some problems ...

- Which installer packages are available for which

platforms…

- An alternative package suggested ... is Subversion

...what are the alternatives?

9


CSEMore thoughts from Huub….

Programming languages:

- Fortran77 vs Fortran90/95/03, C++, C#, Java ...?

- Clearly there may be advantages to higher level

programming languages as they allow the implied

structure of a code to be made explicit through the

use of classes or modules.

- But how well are those languages supported in terms

of tools and compilers,

- how do they compare on run-time inefficiency?

- how do they compare on develop time and maintenance?

- and how do you transform legacy codes.

10


CSEYet more thoughts from Huub….

Scripting languages:

- These languages can be useful to tie various software

components together to build high level applications.

- Currently we are using both Tcl and Python. Of course

there are Perl and Ruby as well.

The nice thing about Python is that it can be extended

with native C-code compiled in dynamically linked

shared libraries without having to rebuild or even

relink the Python interpreter.

Less clear to me is how objects in the libraries

compare with native Python objects when using them in

scripts.

11


CSE...and more thoughts from Huub...

GUI toolkits:

- CSE is developing at least 4 GUIs for working with

atomistic models.

- Unfortunately they are all quite different. Tools

used are AVS, Java, Python/Tkinter, Tcl/Tk, Perl/Tk.

The problem is that Tk is rather limited in the

widgets it offers ...

- Alternatives include Qt, and wxWindows, but how

good are those?

- Above and beyond this GUIs typically require multi-

threading and a object oriented approach. How well

are these supported?

12


CSE…and more….

Parallel programming toolkits:

- Support for parallel programming can come at different

levels. In short there are three different aspects to

parallel programming:

1. communication

2. data distribution

3. algorithms

- MPI offers only support for communication ScaLAPACK

supports communication and algorithms but lets the

user sort out the distribution.

- Global Arrays support all three.

- What else is out there and what is best to use?

13


CSE…and finally...

Management level tools:

- Documentation recommendations

- Programmers guides

- Development processes

- Software testing and validation processes

- User requirements and user process analysis

I think traditionally this is probably the weakest

point within the department...

14


CSESoftware Engineering Support Programme (SESP)

SESP activity to provide and encourage the use of up-to-date software engineering techniques and tools within computational science and engineering. The main goals of SESP are:

– accelerate the introduction and widespread use of high-payoff software engineering practices and technology by identifying, evaluating, and maturing promising or underused technology and practices;

– maintain a long-term competency in software engineering and technology transition;

– enable the UK academic community to make measured improvements in their software engineering practices by working with them directly;

– encourage the adoption and sustained use of standards of excellence for software engineering practice;

– foster collaborations with other groups, in the UK, Europe and the US, that have an interest in the applications of advanced software engineering techniques in computational science.

15


CSEElements of SESP

• Software Quality Assurance

• Processes for Legacy Software

• Evaluation of Methodologies, Tools and Technology

– Integrate Design Environments

– Parallelisation & vectorisation software

– Symbolic Algebra Systems

– Problem Solving Environments (PSE)

– GUIs and user interfaces

– Component technologies

– ...

16


CSESoftware Quality Assurance

• Software Quality Assurance is the basic of software engineering processes that should be undertaken by all software developers.

• The process should include:– Requirements gathering– Design - software and testing– Implementation– Testing– Deployment

• The initial target language for most applications is now Fortran 95 or even Fortran 2003. Although the commercial world of Software QA is dominated by C, C++ and Java, there are good Fortran tools available.

• PlusFORT, ForCheck and the NAG Ware are but three examples for QA tools for use in implementation and testing.

• Clearly CVS is the current tool of choice to track version - but there are others. (Using gCVS and WinCVS can ease the pain)

17


CSEProcesses for Legacy Software

• For many applications within the science and engineering community the root language has been Fortran 77 and for some - even Fortran 66.

• Software engineering has developed and languages have grown and now Fortran 95 and C provide the main modern vehicles for these applications.

• To maintain and continue to develop the science encapsulated in these legacy codes a process of transformation and re-engineering must be formalised.

• This can be broken into three basic steps: – standardisation, – transformation and – re-engineering.

• SESP will develop a process and gather tools to aid this transformation.

18


CSEEvaluation of Methodologies, Tools and Technology

• The computer science community has a long history of developing new methodologies, tools and technologies to aid the development of computing applications.

• These range from new languages, such as C# or JAVA, to frameworks and environments that gather these tools and processes together in an integrated form, such as the Microsoft Visual studio or CodeForge from the Unix world.

• There is a growth in the use of other languages and programming models other than the procedural style of Fortran. Languages such as C++ and Object Orientation are becoming more common in numerical software.

19


CSEDissemination of Software and Results

• All the results of the activity will be disseminated through a CSE Software Engineering Support Programme Web site and through seminars and workshops. The SESP web site will contain:

– An overview of the aims and objectives of SESP– Detail of contacts in the programme– Summary pages on the programmes activities and findings– All technology watch and assessment reports (pdf, ps, html)– Selected software– Links to software and other software engineering pages of interest to

computational scientists.

• Seminars and workshops will be arranged to disseminate the results of the activity or to provide hands on experience with specific software tools. These may be arranged in association with the software vendors.

20


CSESESP Web - www.sesp.cse.clrc.ac.uk

The SESP web site provides access to:

– Information on software tools

– Documentation on the SESP tool set

– Reports & publication on software engineering

– Links to public domain tools that may be of use

21


CSELegacy software - an opportunity?

• Our problem: lots of software from many years of development so what do we do will it?

• Two options:

– throw it away and start again, or

– rescue, transform and reuse.

• We will consider an pragmatic approach that could be taken to retain the functionality of legacy software whilst moving its maintenance and future development processes forward.

• We draw heavily on the work of Charles Norton and Viktor Decyk at the NASA Jet Propulsion Laboratory who have made great strides in developing a systematic approach for the rescue of intellectual property embedded in legacy systems.

22


CSEWhat is Legacy Software?

• Legacy software systems are programs that are:

– still well used by the community

– have some potential inherent value

– but that were developed years ago

– using early versions of languages.

• Often these programs have been maintained and developed for many years by hundreds of programmers, and while many changes have been made to the software, the supporting documentation may not be current and the programming style from the dark ages.

• Legacy software can be characterised informally as old software that is still performing a useful job for the community.

23


CSESoftware replacement?

• Expensive - in staff and time!

• The software represents years of accumulated experience, which is not represented elsewhere, so discarding the software will also discard this knowledge, however inconveniently it is represented.

• The software may actually work well, and its behaviour may be well understood. A replacement system may perform much more badly, at least in the early days. Hence it may be worth recovering some of the good features of the legacy system.

• A typical large legacy software system has many users, who typically have exploited undocumented features and side effects in the software. It may be important to retain the interfaces and exact functionality of the legacy code, both explicit and implicit.

• Users may prefer an evolutionary rather than a revolutionary approach to modernising their software

24


CSEA pragmatic process for legacy software

• To maintain and continue to develop the science encapsulated in legacy codes a process of transformation and re-engineering must be formalised and undertaken.

• This process can be broken into three basic steps:

– standardisation,

– transformation and

– re-engineering.

• The re-engineering step is broken down into four additional steps to help stage the development.

• This follows Norton and Decyk at JPL.

25


CSEA Step-by-step process for legacy software

Legacy Software

Standard-BaseCompilation

Transform software into standard compliance

Undesirable Features

COMMON BlocksImplicit typing#def/#include

Create Interfaces

Wrappers for legacy code

Interfaces for all routines

Add New Capabilities

Dynamic memoryInteroperability

Array Operations

Components & OO

AbstractionIntegration

26



Legacy Software



27


CSEStandardisation

• Get the base code into a conforming form.

• As mention above often legacy codes are in Fortran 77 or 66 or even worst a mixture of all standards, dialects and languages.

• In general the standard of research programming is limited and most often leads to not particularly portable software.

• The developers often adopt mechanisms from other languages.

• The curse of cpp. The # directives for the pre-processor cpp are not in the Fortran standard! #include, #define and #ifdef are three main culprits.

• It is fortunate that most other directives which are added to Fortran programs are expressed as comment lines e.g HPF.

You say: if it compiles on the machine I am interested in - its OK!

28


CSESome examples of typical software

• Program A

– source lines: 16505

– comment lines: 2447

– statements: 11485

– subprograms: 84

– source files: 84

– include files: 6

– error messages: 397

– warnings: 520

– informative messages: 69

• Program B

– source lines:65737

– comment lines:38829


– subprograms: 626



– modules used: 37


– warnings: 2732


29


CSESome examples of typical software (cont.)

• Program C








– warnings: 1964


• Program D






– warnings: 318


30


CSESome typical errors, warnings and information (FORCHECK)

– 88x[ 43 E] illegal characters in front of continuation line

– 16x[ 45 W] too many continuation lines

– 31x[ 46 E] undefined characters at end of statement

– 690x[ 48 E] undefined characters in label field of statement

– 3x[ 53 W] tab(s) used

– 3x[ 60 W] fixed source form used

– 1704x[ 69 E] unrecognized statement

– 853x[ 71 W] nonstandard Fortran statement

– 1x[ 76 E] this statement can only be used within a loop construct

– 3x[ 78 E] statement out of order

– 6x[ 81 E] undimensioned, or statement function out of order

– 1x[ 84 I] no path to this statement

– 7x[ 86 E] program unit END missing

– 5x[ 90 E] unmatched parentheses

– 12x[ 94 E] syntax error

– 259x[ 96 W] obsolescent Fortran feature

– 11x[ 98 W] deleted Fortran feature

– 1x[134 E] missing apostrophe or quote

31


CSESome more..

– 120x[145 I] implicit conversion of scalar to complex– 7x[167 E] illegal usage of subscripts– 1x[183 E] illegal length or kind specification, default assumed– 1x[187 E] illegal to (re)define attribute: object is imported from module– 1x[202 E] multiple specification of data type, this one ignored– 1x[228 W] size of common block inconsistent with first declaration– 3x[241 W] nonstandard mixing of data types in EQUIVALENCE– 72x[287 E] scalar integer constant name expected– 3x[303 E] scalar logical expression expected– 31x[312 E] no value assigned to this variable– 107x[313 I] possibly no value assigned to this variable– 79x[315 I] redefined before referenced– 573x[316 W] not locally defined, specify SAVE in the module to retain data– 373x[319 W] not locally allocated, specify SAVE in the module to retain data– 6x[331 E] illegal usage of substring– 4x[335 E] data type conflict– 25x[340 I] equality or inequality comparison of floating point data– 69x[342 I] eq.or ineq. comparison of floating point data with zero constant–

32


CSE…and some more

– 73x[343 I] implicit conversion of complex to scalar

– 24x[344 I] implicit conversion of constant (expression) to higher accuracy

– 4x[345 I] implicit conversion to less accurate data type

– 4x[348 E] illegal usage of logical operator

– 3x[351 E] illegal usage of operator

– 4x[352 W] nonstandard Fortran 77 operator

– 2x[378 W] pointer not locally associated, specify SAVE in the module

– 5x[387 E] non-matching construct name

– 1x[393 E] missing ENDIF('s)

– 139x[590 E] array versus scalar conflict

– 9x[616 E] input or input/output argument is not defined

– 621x[625 W] nonstandard Fortran intrinsic procedure

– 1x[630 E] illegal argument data type for intrinsic procedure

– 3x[651 I] already imported from module

– 106x[665 I] eq.or ineq. comparison of floating point data with constant

– 6x[675 I] named constant not used

– 3x[691 I] data-type length inconsistent with specification

– 19x[699 I] implicit conversion of real or complex to integer

33


CSETools for conformance checking

• Three main tools have been used:– FORCHECK from Leiden University– NAGWare Tools

• pfort for Fortran 77• nag_modules95 for Fortran 90/95

• These tools are very fussy

• They do not like cpp directives - #ifdef is not Fortran

34


CSECompilers as conformance checkers

• Some compilers have conformance options.

• However all compilers have extensions

• Compilers will pre-process cpp directives

• Good ones for syntax and conformance are:– f95 - Numerical Algorithms Group– ifc (fort) - Intel– lf95 - Lahey

• You should be looking for the “-f95” or “-rigorous” flags

• Include undeclared, un-initialised and unused variables.

35


CSEConditional compilation and pre-processors such as cpp

• Conditional compilation is a very useful feature for developers who need to install their software on multiple platforms.

• The use of a pre-processor as a configuration tool allows the maintenance of multiple configurations in a single file.

• As far as software engineering tools are concerned statements such as #define and #if, that drive pre-processors such as cpp and fpp, are not part of the Fortran language and lead to syntax errors being flagged.

• Even the pre-processor coco now being developed as a component of the Fortran 2003 standard, has the same difficulty.

• Pre-processors provide the programmer with an excellent configuration tool but they are a nightmare to any syntax checking or standard conformance tool.

• Pre-processing with cpp can cause interesting problems on compilation.

36


CSEcpp directives

• For transformation and analysis tools to work correctly cpp pre-processor commands must be remove.

• Basically either the source is pre-processed for one specific configuration, processed and the cpp directives re-introduced as a post-processing step or the effects of the directives must be neutralised before re-engineering but the information they contain preserved in some way.

• Pre-processing with cpp and a suitable sets of -Ds will lead to multiple source - as is generally the intention.

• If the processing is only a standard conformance check or portability check then processing multiple sources is not a problem.

• However if the processing is a transformation of the source merging the transformed sources and re-introducing the cpp directives is a serious problem.

37


CSEWhat can be done?

• Some simple filters are being developed to aid in the assessment of cpp directives and their effect on the transformation tools. These only perform some simple single-pass processing so they are currently very limited their scope.

– cpp-analyse : this script scans the Fortran file and detects cpp directives, e output as diagnostics where possible conflict could arise. These areas are those in which there are variable type specifications, INCLUDE statements and USE within cpp directive block. In general executable statements within a cpp directive block will not present a problem provided the tool processing the software does not move comment statements.

– cpp-remove : this script will remove (comment out) cpp directives and non-executable statements that cause specification conflicts.

– cpp-restore : this script will restore cpp directives within the source on the assumption that the relative position of the comment statement has not moved. This is true in most situations in the body of a source file but may not be true at the file’s head.

38


CSEInitial transformation

• Fortran 77 is a subset of Fortran 90/95

• Initial transformation could be simply from the fixed form of Fortran 77 to the free form of Fortran 90/95 (There are a number of public domain tools which will do this for you - or you could do it yourself with an editor or awk!)

• Automatic transformation is possible once a standard source form is adopted (Some tools are more lenient than others).

• Clearly changing source form gives you little but it can present an interesting and difficult problem - Author Recognition.

39


CSEAuthor Recognition

• An interesting by product of the transformation process is the problem of author recognition.

• Not surprisingly developers and authors have a mental image of their software.

• They are often able to recognise where they are in a code by its appearance and structure of the code on the screen - they have a set of visual bookmarks.

• These help greatly in the development of the software.

• After transformation the developers of the code find that they no longer recognise the software.

• For the developers this may reduce their productivity in development and hence they well be reluctant to use the newer version as the basis of future development.

• Fortunately, in most tools, it is possible to set the control parameters to produce a layout that is not too dissimilar to that of the original.

40



Legacy Software





41


CSESerious transformation

• Once in a standard form automated transformation tools can do:

– Fixed-form Fortran 77 to free-form Fortran 95 - pretty printing.

– Some transformation tools will perform re-structuring of the program:

– Replacing computed GOTOs with IF-THEN-ELSE statements

– Moving COMMON blocks into MODULES

– Moving INCLUDE blocks into MODULES

– Explicitly typing all variable - IMPLICIT NONE only

– Generating INTERFACE blocks for all routines

– Identifying where array syntax can be used

42



• What is not automated as of yet:

– use of pointers/allocatable arrays

– designing and implementing a module structure

– providing dynamic memory allocation

– parallalisation

– vectorisation

– wrapper and encapsulation

– object orientation and user data types

43


CSEAn example of COMMON and INCLUDE

• Subroutine: subroutine sub1

include ’example.inc’

angle=2.0*pi/4.0

c ....

c Body of subroutine

c ...

end

• Include file: real*8 pi,sqrpi,boltz

integer kmaxa,kmaxb,kmaxc,minnode,msbad,mxfix

common/params/kmaxa,kmaxb,kmaxc,minnode,msbad,mxfix,

parameter (pi =3.141592653589793d0,sqrpi =1.7724538509055159d0)

parameter (boltz =8.31451115d-1)

44


CSEAn example of COMMON and INCLUDE (using plusFORT)

• Common & include: MODULE I_example

USE F77KINDS IMPLICIT NONE ! PARAMETER definitions REAL(R8KIND) , PARAMETER :: PI = 3.141592653589793D0 , &

& SQRPI = 1.7724538509055159D0 , & & BOLTZ = 8.31451115D-1

! COMMON /PARAMS/ INTEGER :: KMAXA , KMAXB , KMAXC , MINNODE , MSBAD , MXFIX

END MODULE I_example

• Subroutine: SUBROUTINE SUB1

USE I_example IMPLICIT NONE REAL :: angle angle = 2.D0*PI/4.D0

! Body of subrotuine END SUBROUTINE SUB1

45


CSEINTERFACE blocks

• In many Fortran compilers there have been additional debugging options provided to compare the types and sizes of subprogram dummy arguments against the actual arguments.

• These have provide a very important debugging tool as the correspondence of dummy to actual arguments in a subprogram call harbours many runtime errors - pass an integer into a real and strange thing can subsequently happen.

• Fortran 90/95 provides a mechanism which can eliminate these types of run-time problems by providing the compiler with sufficient information the check the correspondence between actual and dummy arguments.

• In general most interfaces in well designed Fortran 90/95 software will be explicit.

• The compilation system will organise the information to perform actual and dummy argument checking.

46


CSEINTERFACE blocks

• The only problem occurs when a subprogram is defined outside the scope of the current program - most commonly in an external library.

• During compilation the compiler has no automatic information about the subprogram’s interface.

• In using such external libraries the programmer can do one of two things for the compiler. Either:

– declare the subprograms as EXTERNAL or – provide an INTERFACE block.

• The interface to these external subprograms can be provided to the Fortran 90/95 compilation systems through the INTERFACE block.

• By providing an interface block the programmer describes the subprogram’s argument list to the compiler.

• This re-instates the potential for actual and dummy argument checking.

47


CSEINTERFACE block example

FUNCTION lafind(it,nnn,lsi,lsa)

!

!*********************************************************************

IMPLICIT NONE

! Dummy arguments

INTEGER :: it, nnn

INTEGER :: lafind

INTEGER, DIMENSION (nnn) :: lsa, lsi

INTENT (IN) :: it, lsa, lsi, nnn

! Local variables

INTEGER :: i, i1, iz, k

!*** End of declarations rewritten by SPAG

..............................…Body of function

END FUNCTION lafind

48


CSEINTERFACE block example - using NAGWare nag_mkintf95

MODULE nag_mkintf95_mod

INTERFACE

FUNCTION lafind(it,nnn,lsi,lsa)

INTEGER :: lafind

INTEGER, INTENT (IN) :: it

INTEGER, INTENT (IN) :: nnn

INTEGER, INTENT (IN) :: lsi(nnn)

INTEGER, INTENT (IN) :: lsa(nnn)

END FUNCTION lafind

END INTERFACE

END MODULE nag_mkintf95_mod

49


CSEIMPLICIT variable typing

• At the time the IMPLICIT statement and the implicit variable type

conventions must have seemed a good thing.

• All the evidence in software maintenance and re-engineering points to

the fact that IMPLICIT type statement and the implicit typing

conventions obscure and confuse.

• Although the IMPLICIT type statement is still part of the Fortran 90/95

it is clear that the only IMPLICIT statement that really should be appear

in all code is:

IMPLICIT NONE

• IMPLICIT typing should not be used!

50


CSENot all KINDS are the same!

• The use of REAL*8, INTEGER*2 and LOGICAL*1 are all extensions to Fortran 77 and were not part of the formal standard.

• In Fortran 90/95 these can replaced with equivalents: REAL (KIND=DP) or LOGICAL(KIND=1).

• However it should be noted that the KIND definitions are implementation dependent - they are not defined in the standard.

• For example: R8KIND=2 (REAL*8) for the NagWare f95 compiler and for the Intel ifc compiler R8KIND=8 (REAL*8).

• It is a common mistake to assume that:– REAL*8 REAL(8)

• or– DOUBLE PRECISION REAL(8)

• REAL(8) specifies a real variable with KIND=8.

51


CSENot all KINDS are the same

MODULE F77KINDS

! F77KINDS for NagWare f95 compiler INTEGER,PARAMETER :: &

I1KIND = 1, & ! INTEGER*1 I2KIND = 2, & ! INTEGER*2 I4KIND = 3, & ! INTEGER*4 L1KIND = 1, & ! LOGICAL*1 L4KIND = 3, & ! LOGICAL*4 R4KIND = 1, & ! REAL*4 R8KIND = 2, & ! REAL*8 DPKIND = 2, & ! DOUBLE PRECISION CX8KIND = 1, & ! COMPLEX*8 CX16KIND= 2 ! COMPLEX*16

END MODULE F77KINDS

MODULE F77KINDS

! F77KINDS for Intel ifc compiler INTEGER,PARAMETER :: &

I1KIND = 1, & ! INTEGER*1 I2KIND = 2, & ! INTEGER*2 I4KIND = 4, & ! INTEGER*4 L1KIND = 1, & ! LOGICAL*1 L4KIND = 4, & ! LOGICAL*4 R4KIND = 4, & ! REAL*4 R8KIND = 8, & ! REAL*8 DPKIND = 8, & ! DOUBLE PRECISION CX8KIND = 4, & ! COMPLEX*8 CX16KIND= 8 ! COMPLEX*16

END MODULE F77KINDS

These modules were generated automatically using MKKIND.F90 program that is supplied with the plusFORT toolkit from Polyhedron Software

52


CSEVectorisation and parallelisation tools

• Automatic parallelisation has always been the ultimate goal of compiler developers but in the transformation of legacy software to Fortran 90/95 making use of the array facilities in Fortran 90/95 must the initial goal.

• This can be a very time consuming task - identifying the potential inner loops and their associated array variables - identifying any loop level dependencies and recurrence relationships.

• There are very few tools that can aid the programmer in this task and only a few vendors that provide compilers that seek to provide automatic parallelisation.

• One of the few vectorisation tools is VAST/77to90 from Crescent Bay Software [20] is a Fortran 77 to Fortran 90/95 translator.

53



• An example of VAST/77to90’s processing is shown below (taken from the Crescent Bay Software Web site).

• Given the following Fortran 77 input: subroutine demo(a,b,c,n) dimension a(n), b(n), c(n) common /ecom/scratch(10000) do 100 i = 1, n

a(i) = b(i) + c(i) if ( a(i).gt.100.0) then

a(i) = a(i) + scratch(i) go to 100

endifc(i) = a(i)*2 100 continue end

54



• VAST/77to90 created a Fortran 9095 module: module Vecom

real, dimension(10000) :: scratch end module Vecom

• ...and a re-structured main program subroutine demo(a, b, c, n)

USE Vecom implicit none integer n real, dimension(n) :: a, b,c integer :: i a = b + c where (a > 100.0 )

a = a + scratch(:n) elsewhere

c = a*2 end where

end subroutine demo

55



• Once in a standard form automated transformation tools can do:

– Fixed-form Fortran 77 to free-form Fortran 95 - pretty printing.

– Some transformation tools will perform re-structuring of the program:

– Replacing computed GOTOs with IF-THEN-ELSE statements

– Moving COMMON blocks into MODULES

– Moving INCLUDE blocks into MODULES

– Explicitly typing all variable - IMPLICIT NONE only

– Generating INTERFACE blocks for all routines

– Identifying where array syntax can be used

56



• What is not automated as of yet:

– use of pointers/allocatable arrays

– designing and implementing a module structure

– providing dynamic memory allocation

– parallalisation

– vectorisation

– wrapper and encapsulation

– object orientation and user data types

57


CSESoftware Tools

• Some basic tools have been acquired or licensed:– ftnchek (netlib)– FORCHECK (Leiden University)– NAGWare Tools (Numerical Algorithms Group Ltd)– plusFORT (Polyhedron Software Ltd)– Understand for Fortran (Scientific Toolworks Inc.)

58


CSEFORCHECK - Leiden University

Forcheck is is the oldest and most comprehensive Fortran verifier on the market.

– It performs a static analysis of Fortran programs or separate Fortran program units.

– Generally Forcheck detects more anomalies in your program than most compilers do.

– Forcheck is ideally suited to get a fast insight in existing and legacy programs.

– It composes documentation of your project with cross-reference tables of each program unit, the complete program and produces a call-tree.

– Forcheck can be used as a software engineering tool in the various stages of the development process and can verify the conformance to each level of the Fortran standard.

– Beside the full Fortran syntax Forcheck supports many language extensions of all popular compilers by compiler emulation.

59


CSEFORCHECK - Leiden University

Forcheck features (cont.)

– Supports many language extensions of all popular compilers - has the concept of compiler emulation.

– FORCHECK is fully configurable so you can tune the analysis and output to your needs.

– Stores the global information of the analysed program-units in libraries which can be referenced in subsequent FORCHECK runs to verify the consistency of all references and common-blocks.

60


CSEplusFORT - Polyhedron Software Ltd

plusFORT is a suite of tools for Fortran programmers. The main

components are summarized below:

– SPAG - The primary analysis and restructuring tool of plusFORT. SPAG

processes Fortran 77 with all common extensions, and almost all Fortran

90/95. It can also translate Fortran 77 to Fortran 95

– GXCHK - A global static analysis tool

– CVRANAL - A coverage analysis reporting tool

– QMERGE - A version selection tool

– QSPLIT - A small file-splitting utility

– AUTOMAKE - A tool for minimal recompilation of Fortran (66, 77, 90 and

95) and C programs.

61



SPAG is a multi-purpose tool for analysing and improving Fortran programs. It combines restructuring and re-formatting with translation to Fortran 95, and both static and dynamic analysis in a single package.

SPAG can (by setting 150 or so options in its configuration file!)– restructure 'rats-nest' Fortran 66 to modern and structured Fortran (77 or

95), or apply a final polish to well written modern code. – create Fortran 95 modules to replace Fortran 77 COMMON blocks and

INCLUDE files. – create Fortran 95 interface blocks to facilitate compiler argument checking.– insert explicit type declarations for IMPLICITly typed variables, or rewrite

the declarations from scratch using either Fortran 77 or Fortran 95 declaration style.

– can translate VAX structures to Fortran 95 derived types.– identify, and optionally remove unused variables and code fragments.– change variable names in a simple and safe way.

62



– convert back and forth between Fortran 77, and code with Fortran 95 extensions such as DO WHILE, ENDDO, CYCLE, EXIT and SELECT CASE.

– SPAG can also convert between the new and old source forms, and the new and old declaration styles.

– make re-formatted code even clearer by using upper and lower case to distinguish different types of symbol.

– insert probes to perform a dynamic analysis of your program as it runs. Dynamic analysis can identify many bugs which are not apparent to a static analyser.

– use dynamic analysis to identify variables by their contents at run-time. – insert probes to identify untested code and computational hot-spots. – write out symbol table files for reference, or for input to other QA and

analysis programs, such as GXCHK– understands Fortran 77, VAX Fortran, and almost all Fortran 95, together

with many common extensions, such as Cray pointers, ENCODE/DECODE etc.

63


CSENAGWare Tools - Numerical Algorithms Group Ltd

• The NAGWare Fortran Tools provide users with the ability to analyse and transform Fortran 77,Fortran 90 and Fortran 95 codes.

• These tools can be used in a range of ways:– Quality Assurance Standardisation Enforcing coding standards– Porting to new platforms– Converting from fixed format Fortran 77 to free format Fortran 95– Normal day-to-day development

• The NAGWare Fortran Tools suite consists of the following components:

– NAGWare Fortran 95 Tools– NAGWare Fortran 77 Tools

64


CSENAGWare Tools - Numerical Algorithms Group Ltd

NAGWare f95 Tools• Analysers

– nag_coverage95

– nag_depend95

– nag_fcalls95

– nag_modules95

– nag_xref95

• Transformers– nag_cbm95

– nag_chname95

– nag_decs95

– nag_mkintf95

– nag_polish95

– nag_polopt95

– nag_prest95

– nag_struct95

– nag_ustan95

NAGWare f77 Tools• Analysers

– nag_fcalls

– nag_fxref

– nag_libdoc

– nag_metrics

– nag_pfort

• Transformers– nag_apt

– nag_chname

– nag_decs

– nag_lvi

– nag_polish

– nag_polopt

– nag_profile

– nag_struct

65


CSEThe QAPortal - www.qaportal.cse.clrc.ac.uk

• The QA Portal provides a simple web interface to a number of the common software engineering processes

– analysis

– transformation.

• It is built on a standard client/server model with user registration and password protection.

• Will process Fortran 77 and Fortran 90/95 software.

• Single files or archive (tar, zip) collections can be processed.

• The results can be view or saved to a local file.

• Documentation on the QA Portal and the associated tools is provided online.

The User

Web Server

QA Server

ssh/scp

http

66


CSEUnderstand for FORTRAN - Scientific Toolworks Inc.

• Understand for FORTRAN is an interactive development environment (IDE) tool providing reverse engineering, automatic documentation, metrics and cross referencing of FORTRAN source code.

• It supports FORTRAN 77 (F77) and FORTRAN 90 (F9X) language standards, with common VAX, Cray and Salford extensions.

• Understand for FORTRAN helps you reverse engineer, understand and maintain large amounts of legacy FORTRAN source code.

• The tool includes a syntax colourizing "smart" editor that can tell you about what you are editing.

• It also includes numerous graphical reverse engineering views designed to help you understand and assess changes you are considering in your code.

67


CSEA look at some tools...

• FORCHECK - Leiden University

• Understand for Fortran

• Visual Fortran

• WinCVS

68



Legacy Software





Create Interfaces

Wrappers for legacy code

Interfaces for all routines

Add New Capabilities

Dynamic memoryInteroperability

Array Operations

Components & OO

AbstractionIntegration

69


CSESummary

• SESP will provide CSE with software engineering tools and expertise which, to some extent, will be driven by user needs.

• A process for migrating legacy Fortran software has been defined and some software tools identified.

• The legacy process tested on a number of applications with reasonable results - much more automation is required.

• A software tools resource has been started - needs expanding.

• A web interface to some of the tools has been written - this need to be expanded and more flexible.

• There are software tools to aid migration of codes - conformance to a standard source form is the penalty (not really a penalty).

• There are tools to help in the understanding and documentation of software - a short learning curve is required.

70


CSEComments, questions & discussion

• What are your requirements?

• How do you think they would be best meet?

• What areas would you like SESP to address?

• ...

computational science & engineering department cse 1 software engineering tools – can they be...

Documents

software engineering

cse science programme

quality software

cse perspective

excellent science

software developers

software skills

ccp requirements