Reverse Engineering Legacy Interfaces:An Interaction-Driven Approach

E. Stroulia, M. El-Ramly, L. Kong, P. Sorenson B. MatichukComputing Science Department CEL Corporation

University of Alberta 9637 45 AvenueEdmonton, AB T6G 2H1 Edmonton, AB T6E 5Z8

Canada Canadafstroulia,mramly,lanyan,sorensong@cs.ualberta.ca bmatichuk@celcorp.com

Abstract

Legacy systems constitute valuable assets to the organi-zations that own them. However, due to the developmentof newer and faster hardware platforms and the inventionof novel interface styles, there is a great demand for theirmigration to new platforms. In this paper, we present amethod for reverse engineering the system interface thatconsists of two tasks. Based on traces of the users inter-action with the system, the “interface mapping” task con-structs a “map” of the system interface, in terms of the in-dividual system screens and the transitions between them.The subsequent “task and domain modeling” task uses theinterface map and task-specific traces to construct an ab-stract model of a user's task as an information exchangeplan. The task model specifies the screen transition dia-gram that the user has to traverse in order to accomplishthe task in question, and the flow of information that theuser exchanges with the system at each screen. This taskmodel is later used as the basis for specifying a new graph-ical user interface tailored to the task in question.

1. Introduction and Motivation

Legacy systems constitute valuable assets to the organi-zations that own them. Very often, a legacy system is thesole repository of valuable corporate knowledge collectedover a long time and the logic of the organization's busi-ness processes. Unfortunately, there is usually little or nosupporting documentation for these systems, which makestheir maintenance, evolution and integration with other ap-plications very difficult. In addition, the text-based inter-faces of these systems offer very limited ways to organizeand present information to the user, and therefore they are

usually non-intuitive and difficult to understand. This factmakes the training of new personnel a challenge. Further-more, the proprietary nature of these systems prevents thecorporation from making use of the opportunities that theWWW presents today and from offering direct access to itsinformation system to its partners and customers. Thus, theproblem of wrapping and/or migrating legacy systems [9]has emerged as one of the most interesting and challengingones in software engineering.

To that end, one might attempt to understand the systemdesign in terms of the data structures and control of pro-cessing by examining its code [10]. However, legacy sys-tems tend to be rather large and complex. In addition, in alllikelihood, the principles of their original design have beencompromised with layers of “glue code”, that is, code formodifications unrelated and perhaps contradictory to theiroriginal design. Even worse this code has been developedby people not involved in the original design process andtherefore unaware of the design rationale. Furthermore,in some cases the code is only available in its executableform. For all these reasons, this approach may be pro-hibitively costly or even impossible.

An alternative approach, which we have been investi-gating in the CELLEST project, is to try to understand theinformation and the process logic that a legacy system em-bodies by reverse engineering the system interface. In gen-eral, transaction-based systems developed for mainframes,whose purpose is to support tasks such as data entry,database querying and report generation, do not performlong computations but interact often with their users to getinput or return the output of their transactions. As a result,their interfaces expose to their users a lot of the informa-tion stored in their internal repositories and its organiza-tion. They also correspond quite faithfully to the differenttasks that they are currently used to perform. Therefore,

examining how the system is being used, based on how itsusers interact with it, can potentially provide an alterna-tive approach to understanding the processes that the sys-tem currently performs in the organization. Clearly, if theend goal is to extend or modify the system functionality bymodifying its current code, then such interaction-based un-derstanding is insufficient because it provides only a modelof the user tasks that the system supports and it completelyignores low-level design decisions such as data structuresand algorithms. If on the other hand, the overall purposeis to migrate the system interface to newer platforms orto wrap it with well-defined APIs to enable its interactionwith other systems, then this alternative interaction-basedunderstanding approach is suitable.

In this paper, we present a method for reverse engineer-ing the system interface that consists of two tasks. Basedon traces of the users' interaction with the system, thein-terface mappingtask constructs a “map” of the systeminterface, in terms of the individual system screens andthe transitions from one screen to another. A screen is adistinct unit of information presentation, available to theusers at a particular point in the process of their interac-tion with the system. A transition from one screen to an-other is performed by a sequence of user actions, such askeystrokes and cursor movements and may be conditionedupon a specific internal system state. The subsequenttaskand domain modelingtask uses the interface map and task-specific traces to construct an abstract model of a user'stask as an information exchange plan. The task modelspecifies the screen transition diagram that the user has totraverse in order to accomplish the task in question, and theflow of information that the user exchanges with the sys-tem at each screen. We also discuss how this task modelcan later be used as the basis for specifying a new graphicaluser interface tailored to the task in question.

The rest of the paper is organized as follows: Section2 describes the overall architecture of theCELLEST envi-ronment that we are developing for implementing our in-terface reverse engineering method. Section 3 describesour method for mapping the system interface, and section4 discusses the experiments we have performed to date toevaluate it. Section 5 describes our method for task anddomain modeling. Section 6 discusses some of the relatedresearch literature and outlines our current directions forfurther research. Finally, section 7 summarizes the paperand offers some preliminary conclusions.

2. The CELlest Architecture

The CELLEST system (see Figure 1) interacts with thelegacy system through two different middle-ware tools, de-veloped by CEL corporation [16]: the Recorder and the Pi-

lot. The Recorder is a component emulating the protocolof communication between the mainframe where the soft-ware resides and its terminals1. Thus it enables a user tointeract with the system in the same way that a terminalwould, while at the same time, it records the interactiontrace. The Pilot is a component that enables the controlof the interface from an external application, by “translat-ing” events in other applications to corresponding controlsequences to the legacy interface.

The overall process of theCELLEST system uses asinput a set of system-user interaction traces, collected bythe Recorder. A trace, as captured by the Recorder, con-sists of a sequence of “screen snapshots”, i.e., copies of thescreen buffers that the user has interacted with, and “actionrecords”, i.e., records of all the keystrokes that the user hasentered to the system while at a particular screen. A col-lection of traces from the daily usage of the legacy systemis the input of the interface-mapping process.

The interface-mappingtask (T1 in Figure 1) is accom-plished by the LENDI system (LEgacy Navigation Do-main Identifier). As the user interacts with the system,the Recorder collects the snapshots of the system screensthat the mainframe sends to the user terminal. Then, theinterface-mapping process proceeds to recognize identify-ing features in the screen snapshots of the trace, in orderto cluster several screen snapshots together as instances ofa single unique system screen. The output of this processis a directed graph, henceforth called theinterface graph.The graph nodes correspond to the individual screens ofthe system and its edges correspond to the user action se-quences that enable the transitions of the system from onescreen to another. Each node is characterized by a predi-cate that evaluates whether a particular screen snapshot isan instance of the screen that corresponds to the node. Anedge may be annotated by a condition, if the transition itrepresents is only possible in a particular system state.

To provide sufficient information to map the system in-terface, the trace should at least contain screen snapshotsof all the different tasks that the system users perform, andeachnon-deterministic task should be performed at leastas many times as the number of its distinct screen navi-gation paths. To put it more simply, the collected tracesshould exhibit all the potential behaviors of the interface.For example, in order for the interface graph to reflect thefact that the user can return to the “login screen” from any-where in the system by pressing “@12”2, the user shouldhave performed this action, i.e., hit PF12 to return to thelogin screen, on all screens at least once.

1Currently the Recorder is able to emulate a variety of protocols in-cluding 3270, AS400, vt100 etc. To date, however, in theCELLESTproject we have focused exclusively on the 3270 protocol [1]

2We use the symbol @ to denote function keys, e.g., “@12” is functionkey PF12.

Figure 1. The CELLEST environment

The second reverse-engineering task,task and domainmodeling(T2 in Figure 1), is performed by theURGENTsystem (User interface ReGENeration Tool).URGENTuses as input a collection of task-specific traces mappedonto the interface map. Based on them, it constructs anabstract model for the task in question.

URGENT adopts an “information exchange” approachfor modeling a user's task. According to this model,the users' interaction with the system allows them toprovide (obtain) some pieces(s) of information to(from)the system. The users accomplish their information-exchange purposes by performing a sequence of elemen-tary information-exchange actions on individual systemscreens. We have identified two types of such elementaryactions:

� When the users provide information to the system,such as with data entry operations, then they are con-sidered to performtell actions.

� Alternatively the users may obtain information fromthe system with anask action. The system users in-dicate such actions toURGENT by highlighting thescreen area where the interesting information appears.

– If the information in question always appears in

the same area, then this is anask-standard ac-tion, because the user is always interested in thesame standard screen area;

– if, on the other hand, the screen on which the askaction is performed is dynamic and the informa-tion of interest may appear in different positionsin the screen, then this is anask-select action,because with every individual screen buffer theuser has to “select” which screen area to high-light.

In addition to classifying the user's information-exchange actions according to the direction of the infor-mation flow, URGENT also classifies the different piecesof information exchanged according to their scope. So, apiece of information “asked” or “told” by the user can bea System Constant, aUser Variable, aTask Constant, or aProblem Variable.

� System Constantsare constant strings whose valuesare independent of the user's task when visiting thesescreens.

For example, in systems with multiple complex sub-systems the user may have to select one among themby entering its name in some particular screen. Thesepossible choices are all system constants.

� User Variablesare data items associated with the in-dividual users performing the task.

For example, a user's login and password are uservariables. Also, in the above example of system con-stants, if a particular user always works with a singlesubsystem, then this selection is also a user variable.

� Task Constantsare constant strings that need to be en-tered on a screen when it is visited in service of a spe-cific task.

In menu-driven systems for example, the user, in or-der to perform a specific task, usually has to make asequence of selections on the menu tree. All the ele-ments in this sequence are task constants.

� Problem Variablesare data items that flow throughthe screens of a task session; they are either originaluser input, or intermediate system output used as in-put in subsequent screens, or system output used astask output.

Based on these two orthogonal classification dimen-sions, URGENT produces a domain model of the system,in terms of the unique pieces of data that the system ex-poses to the user, and a task model, in terms of well-definedsequences of information-exchange actions performed onsubsets of that data.

Finally, in a forward-engineering phase (T3 and T4 inFigure 1), based on the abstract task model and a chosenuser profile, URGENT specifies and generates a graphicaluser interface. This new interface can properly convey alltypes of information that need to be exchanged betweenthe user and the system in service of the task at hand. Theabstract task model constitutes a bridge between this newlydesigned GUI and the actual system interface, since the dif-ferent operations on the GUI are designed to implement theinformation exchange actions that the users perform on theoriginal interface for their tasks. At this point, the graphicalinterface generated byURGENT is integrated with the Pi-lot that drives the legacy system by issuing the keystrokescorresponding to the user's actions to the underlying legacyinterface.

3. Interface Mapping

As we have already discussed, the first phase of theCELLEST process is the mapping of the system interface,that is, the construction of the interface graph. This pro-cess must address two different problems. The first is theidentification of the individual system screens, that is, theinterface graph nodes. This is the problem on which wehave focused on to date, andLENDI's screen identifica-tion method is described in detail in the next subsection.

The second problem involves learning the interface graphedges, i.e., the possible transitions between screens andtheir preconditions. Although we have not yet addressedthis problem, some initial ideas are sketched out in subsec-tion 3.2.

3.1. Screen Identification

Legacy screens range from very static to very dynamic.Some screens appear always the same irrespective of theuser's history of interaction with the system. Consider, forexample, login screens or screens describing a menu ofpossible actions. Such screens are generally poor in dataand their purpose is to enable the users to control their sub-sequent interaction with the system. On the other hand,other screens can be very dynamic. Consider a screen pre-senting the result of a user query. The purpose of such ascreen is to present data, and depending on whether thepresented data is well-structured or not, the different in-stances of the screen may share a great or a very smalldegree of commonalities. For example, if the query resultis a set of data records, the screen may always appear or-ganized as a table of data, although its actual content willbe different. If, on the other hand, the result of the query isfree text, then the appearance of the screen will vary more.

3.1.1. Screen Features. To develop a screen-identification process able to cope with this diversity ofscreen types, we have identified a set of different featuresthat belong to three different families. The first familyconsists of features based on commonly used strings insome special screen areas, such as codes, titles, date andtime, and page numbers that appear in the periphery of thescreen.

� Screen Code:In some systems, a sequence of al-phanumeric characters can be found usually at oneof the screen corners. When such a code exists ona screen, it usually is a very discriminative feature.

� Screen Title:Quite often screens have titles, usually inone of the top two lines. The screen title is a sequenceof words, usually describing the purpose of the screen.As with codes, when they exist, titles can potentiallyuniquely identify a screen.

� Date, Time and Page Location:The format and loca-tion of date, time and page information, usually foundon the left or right of the first two lines of the screen,can also be a discriminative feature.

The second family of features is intended to recognizegeometric patterns in the placement of the information onthe screen.

� Field Positions: The 3270 protocol [1] provides adescription of the organization of each presentationspace pushed to the terminal in terms of “fields”.These fields – their number and locations – providevaluable information about the overall screen organi-zation. Especially for static screens, it is usually thecase that the field locations are the same for all theinstances of a single screen.

� Projection Profiles:Projection profiles, widely usedin document analysis [13], are a mapping from a two-dimensional image to a waveform, whose values arethe sums of the values of the image pixels along somespecified dimension. A projection profile is obtainedby determining the number of black pixels that fallinto a projection axis. A deep valley in the profilewith a certain predefined width is called a cut. Byconsidering the legacy screen as an image, with eachcharacter as a pixel, we have utilized this techniqueto extract some useful features. For a legacy screen,a projection profile is a histogram of the occurrencesof a particular character (or set of characters) on thescreen. Depending on the character(s) counted, a vari-ety of different profiles can be collected. For example,horizontal and vertical profiles of all characters revealtable, list or structured report layout, horizontal num-bers profile can indicate enumerated lists that are of-ten used for presenting menus in legacy systems, andthe profiles of special characters, such as�, , j, areuseful for identifying patterns of separator lines.

Finally, a set of application-specific features can be ex-amined.

� Keywords: The existence and locations of particularkeywords or sets of keywords can help in screen iden-tification, e.g. Menu, Report, Form, Help, File, In-put, etc. Also, keywords likeError, Invalid, no/Not,match/found/valid/available, etc. help in identifyingerror screens or error state. Others likePage, nn of nn(where nn is a number),continued, etc. suggest thatthe screen is an instance of a sequence of the samescreen, or one page of a multi-page screen.

� The Cursor Label and Position:Along with everynew 3270 presentation space sent to the terminal, thelocation of the cursor on screen is also sent. Thisinitial cursor location and the label to its left can besignificant in identifying the screen in command-linedriven systems.

3.1.2. The Screen Recognition Process.The screenrecognition process requires the configuration of a set ofrecognizers, Ri; 1 � i � n. Each recognizer employs one

or more of the features discussed above, in order to evalu-ate whether two screen snapshots are instances of the samescreen. A recognizer compares two snapshots in terms ofits features. If the two snapshots share a feature then thisfeature votes that the snapshots are instances of the samescreen. The recognizer votes that two screen snapshots areinstances of a single screen with some probability, com-puted as the weighted sum of all the votes of the individualfeatures of the recognizer.

Vi =P

j=1::# of features in Rivj �wfi;jg;

whereVi is the vote ofRi, the ith recognizer,vjis the vote of thejth feature for this recognizer,andwfi;jg is the weight with whichjth feature con-tributes to the overall vote of theith recognizer.

If a feature is not found on a particular screen, the recog-nizer may be configured to either ignore it, or to considerits absence as evidence that there exists a distinct screenthat lacks the feature in question of which the two snap-shots are instances.

Let us now discuss how the overall recognition processworks, that is, how the list of the interface graph nodes isconstructed. The interface graph is initialized to be empty.Subsequently, each screen snapshot in the trace is accessedand compared to all the previously recognized nodes bythe configured recognizers. Each recognizer compares thecurrent snapshot against a representative of each previouslyrecognized node, and it computes the probability that theymatch, i.e., that they are instances of the same screen. Thefinal probability that the new snapshot matches a previ-ously recognized node, that is that it is an instance of thescreen that the node represents, is the weighed and normal-ized sum of the votes of the different recognizers.

Pi=1::#ofrecognizersfVi �wig,

whereVi is the vote of theith recognizer, andwi isits weight relative to the other recognizers.

If the most probable match is above a preset threshold,then the current screen snapshot is considered to be an in-stance of the same screen as the previous screen snapshotwith which it was compared. Otherwise, the current screensnapshot is considered to be an instance of new screen,never encountered before, in which case a new cluster isintroduced. At the end, the process has constructed a setof snapshot clusters. Each cluster contains snapshots thatare instances of the same screen, and therefore correspondsto a node in the interface graph. The combination of fea-tures, on which a cluster of snapshots match, constitutesthe identifying predicate of the node.

3.1.3. Setup Advice. To support the configuration of therecognizers, LENDI has a heuristic advisor component.The advisor takes as input a list of the features to be usedfor the screen recognition process and an estimate of thenumber of unique screens in the system, that is, the numberof interface graph nodes.

The advisor performs a complete recognition process onthe available traces using every single feature in the inputlist and every possible OR-combination of two features.Then the advisor considers only the recognizers that re-sulted in a number of output clusters between 50% and100% of the number estimated by the user. The advisorthen suggests thateach feature contained in the selectedrecognizers be used in a separate recognizer, whose weightwill be analogous to the number of occurrences of this fea-ture in all the selected recognizers. Finally, the user tunesthe advisor's suggested configuration and determines thethreshold value.

3.2. Transition Identification

The second problem in the interface mapping task is tolearn the interface graph edges, i.e., the possible transitionsbetween screens and their preconditions. Transitions aremade possible by user actions on a screen. An action isa sequence of cursor movements and keystrokes that endsby an Attention Identification key (AID [1]), such as En-ter, or a function key, etc. Different types of actions existin different systems or even within one system. For exam-ple, in some legacy systems, the final function key of thekeystroke sequence drives the navigation. In such systems,different function keys correspond to different transactions– much like in a menu – and the user enters the data re-quired on the screen and uses a function key to select thetransaction to be performed on this data. Other systems arecommand-driven, and require the user to type in a com-mand, usually followed by the Enter key. Often, differentforms of the command are available, for example, a com-mand string and all its prefixes may have the same effectin some systems. Yet other systems are driven by a stringof data items entered at particular location(s) on the screenfollowed by the Enter key.

Transition preconditions can also be of various types.Some times, they may be observable on the screen itself inthe form of a system message. For example, a transitionfrom one data entry screen to another may be possible onlyif all the fields are entered; in case of incomplete entries anerror message may appear on the screen that prevents allbut the error-handling action, i.e., the filling of the emptyfields. In other cases, a transition may be conditioned uponthe value of a piece of data entered on the same (or someprevious) screen(s). For example, only if the value entered

in the “age field” is above 60, can the user access a screenpresenting pension information.

This variety in action types and their preconditionsmakes the problem of edge recognition quite challenging.An especially interesting aspect of the problem is that ofcompleteness, that is, how to recognize all the transitionsof the system screens. We are currently starting to workon this problem and our approach is to try to formulatehypotheses about the screen behavior based on its layoutand/or the examples of its transitions contained in the trace.Then, based on these hypotheses we generate predictionsabout the types of the expected screen transitions. Suchexpected screen transitions can guide the understanding ofthe instances of the transitions that exist in the trace, andcan also provide the basis for an exploratory phase wherethe screen behavior is tested with novel action sequences,not seen in the trace. For example, some screens containa line – usually at the bottom – listing different functionkeys followed by a string indicating their functionality onthe given screen. If such a line is found on a particularscreen, then the expectation is that each individual func-tion key listed corresponds to a transition from this screento another. If the trace does not contain examples of allthese transitions, then the interface can be tested by apply-ing the function keys that were not exercised in the trace tothe screen in question. Similarly, a screen may be hypothe-sized to be a data-entry form, due to the substantial numberof user keystrokes in the example transitions of the screenin the trace. Then that screen can be tested with severalalternative action sequences, generated with modificationsto the action sequences that exist in the trace, to explore itsbehavior with erroneous or incomplete input.

In general, it is not possible to guarantee the complete-ness of the interface map with respect to the actual systeminterface. However, we envision a background monitoringprocess which will recognize new transitions, not specifiedin the interface map, and will incrementally – and possiblyinteractively – update the map to more closely reflect thesystem.

4. Evaluation

LENDI's screen identification process was tested withtwo legacy systems. The first was an insurance system andthe second was the Library Of Congress Information Sys-tem (LOCIS) [18]. The results of these experiments arereported in the next paragraphs.

4.1. First Experiment

The trace used for the first experiment – with the in-surance system – is 371 screens long. It has 106 different

screens. Some screens appear once and some exist as manyas 14 times. Some screens, e.g. menus, are very static andothers, e.g. free report forms, are very dynamic. Most ofthe features described above were extracted. In particular

1. a1 encodes a classification of left, middle and rightareas of the top two lines of the screen snapshot intoscreen code, title, date, time, page information, orblank

2. a2 is the location of code and title on screen snapshot,if any

3. a3 is the text in the middle of the first line4. a4 is the text in the right of the bottom line5. b1 encodes the locations of the 3270 fields6. b2 encodes the number of input fields on the screen

snapshot7. c1 encodes the cursor's label and initial location8. d1 andd2 encode the horizontal and vertical all char-

acters profiles respectively9. d3 encodes the horizontal numbers profile

10. d4 encodes the vertical words profile11. d5 encodes four vertical profiles for four special char-

acters: , �, ( and: .

Two experiments were performed with this system: inthe first one, aLENDI user manually configured the rec-ognizers, while in the second one, the advisor proposed therecognizers' configuration.

Manual Setup: In this experiment, aLENDI user re-viewed the trace thoroughly to decide the most useful, i.e.,the most discriminating features for screen identification,and then, manually configured the recognition setup. Af-ter performing the identification process, the user reviewedthe results for errors, and fine-tuned the setup to avoid theseerrors. It took ten such recognition/review/reconfigurationrounds to reach the setup shown in Table 1.

Table 1. Experiment 1: Manual recognitionsetup

Weight Features Value Ignore ifEmployed Empty

1 1 a4 100 Y2 1 a1,a3,a4 10,45,45 Y,Y,Y3 1 d1,a1,a4 10,45,45 Y,Y,Y4 1 d2,a1,a4 10,45,45 Y,Y,Y5 1 d5,a1,a4 10,45,45 Y,Y,Y

The final setup included five recognizers. The weight,features used, value of each feature and the significance ofthis feature if empty are shown in columns 2 to 5 of Table

1. The threshold used was 20%. The output of this finalsetup was 110 different identified screens. It included nofalse positive errors and 4 false negative errors, i.e. 1.1%.

A discussion is necessary here on the consequences ofthe different types of errors. In this problem, false posi-tive matches between snapshots are graver errors than falsenegatives, because, they collapse snapshots with poten-tially different behaviors within a screen cluster. In com-parison, false negative errors differentiate among same-screen snapshots thus resulting in two nodes in the inter-face map corresponding to the same screen. The formererror type constitutes an actual misrepresentation of theunderlying interface. When an external application usesthe interface map to plan a screen traversal sequence, sucherrors in the map can cause incorrect predictions about theresult of actions in the erroneously collapsed nodes and cantherefore result in the external application “getting lost” inthe legacy interface. The latter type of error, where snap-shots of the same screen are erroneously split into morethan one clusters, simply constitutes a redundancy in theinterface model.

Advisor's Setup: In this experiment, the advisor was ini-tialized to propose recognizers based on featuresa1, a3,a4, c1, d1, d2, d4 andd5 and their combinations such thata target number of 100 screens would be recognized fromthe input trace. The advisor suggested the setup shown inTable 2 in normal font.

Table 2. Experiment 1: Automatic recogni-tion setup

Weight Features Value Ignore ifEmployed Empty

1 4 a1, a4 10, 90 Y, Y2 2 a3 100 Y3 6 user

�! 15 a4 100 Y4 1 c1 100 N5 2 d1 100 Y6 2 d2 100 Y7 3 d4 100 Y8 1 d5 100 N

After reviewing the results, theLENDI user made thechanges shown in bold in Table 2 and also suggested athreshold of 35%. The output of this setup was 113 differ-ent identified screens. There were no false positive errorsand 7 false negative errors i.e. 1.9% of the total number ofscreen snapshots were misclassified as unique.

4.2. Second Experiment

In the second experiment,LENDI was tested with atrace taken from the 3270 on-line connection of LOCIS.The trace represents catalogue browsing and help tasks. Itis 200 screens long. It has 48 different screens. 32 of thesescreens are static, e.g. help or information screens andmenus. The others vary in how much dynamic they are.Some screens exist once and other exist up to 20 times.

The features extracted are the same as the ones extractedin experiment one with the following changes

1. a3 is the text at the left of the first line2. c1 encodes the cursor's label only3. d5 encodes two horizontal profiles for * and -

Manual Setup: The user of LENDI used featuresa1,a3, a4, c1 and d5 only. He needed eleven recog-nition/review/reconfiguration rounds to reach the setupshown in Table 3.

Table 3. Experiment 2: Manual recognitionsetup

Weight Features Value Ignore ifEmployed Empty

1 1 a1,a4,c1 10,45,45 Y,Y,N2 1 a3,a1,a4,c1 10,30,30,30 Y,Y,Y,N3 1 a4 100 Y4 1 c1,a1,a4 10,45,45 N,Y,Y5 1 d5,a1,a4 10,45,45 Y,Y,Y

The threshold used was 20%. The output of the finalsetup was 49 different identified screens. It included 2 falsepositive errors (1%) and 6 false negative errors (3%). Fiveof the false negative errors happened due to misclassifi-cation of instances of a system message screen. Despiteof having the same behavior, the appearance of these in-stances may look completely different.

Advisor's Setup: In this experiment, the advisor was ini-tialized to propose recognizers based on featuresa1, a3, a4ndc1 and such that a target number of 40 screens would berecognized from the input trace. The advisor suggested thesetup shown in Table 4 in normal font.

After reviewing the results, theLENDI user made thechanges shown in bold in Table 4 and also suggested athreshold of 20%. The output of this setup was 51 dif-ferent screens. There were 2 false positive errors (1%) and7 false negative errors (3.5%).

Table 4. Experiment 2: Automatic recogni-tion setup

Weight Features Value Ignore ifEmployed Empty

1 2 a1, a4, c1 20, 40, 40 Y, Y, N2 2 a3 100 Y3 3 a4 100 Y4 2 c1, a1, a4 20, 40, 40 N, Y, Y

We find these preliminary results encouraging and weare in the process of conducting further experiments withtraces of different lengths from several different systems.Although not completely automated yet, the screen recog-nition process is quite effective. We expect that use of ac-tion information will help to further automate the processand increase its precision.

5. Task and Domain Modeling and InterfaceMigration

After an interface map has been constructed, the taskbecomes to better understand the tasks that the users per-form with the system and the types of information that thesystem exposes to them. TheURGENT process for con-structing an abstract task model and, based on it, designinga GUI consists of three steps:Task and Domain modeling,Abstract User Interface SpecificationandGraphical UserInterface Generation[5]. These steps will be illustratedin the subsequent subsections in terms of a hypotheticaltask in an Insurance Information System. Consider a situa-tion where the insurance company computerized its claimsdepartment separately from their customer's database, andtherefore owns two separate subsystems, i.e.,subsystem1andsubsystem2, containing their customer and claims in-formation. Suppose further that insubsystem2, the usersmust enter the customer's claim number in order to retrievethe data relevant to generating a report on the customer'saccident claim. If they only know the name of the cus-tomer, they have to first search for the claim number in thesubsystem1by entering the customer name, before they cango intosubsystem2to retrieve the report data.

5.1. Interactive Task and Domain Model Construc-tion

The input to this task of task and domain modeling is aset of traces of users performing thehypothetical task de-scribed above, i.e., entering customer names to get their

Table 5. Part of a Recorded Trace and its Analysis

Screen Name Data items 1st phase 2nd phase 3rd phaseSingle trace Multiple traces Different users

Signon “lanyan”@T Tell var2 Task-specific User variable“t65j”@E Tell var3 Task-specific User variable

Name search “scott”@E Tell var6 Problem-spec. Problem variable

Claim retrieval mouse(3,12)to(3,17) Ask var7 Task-specific Ask standardProblem Variable

Menu “7889”@T Tell var7 Problem-spec. Problem variable“a1”@E Tell var8 Task-specific Task Constant

corresponding claim numbers and using these numbers toretrieve the information about theiraccident to generate areport. The overall purpose of the task analysis is to iden-tify the sequence of actions that the users perform to ac-complish their task, as well as the pieces of information ex-changed between the user and the system and their scope.Task analysis occurs in three phases. The first examines asingle trace of the task, in order to create a baseline sub-graph of the overall interface map that the user traversesto accomplish the task in question, and the pieces of infor-mation exchanged during the traversal. The second phaseexamines multiple traces of the same task performed bythe same user, in order to categorize the information ex-changed between the user and the system in system con-stants and problem variables. Finally, the third phase ex-amines multiple traces of the same task performed by dif-ferent users, in order to differentiate between system con-stants and user variables. The first two columns of Table 5describe a part of a recorded session, in terms of screensvisited and data entered and obtained, for the above re-porting task. The variables in quotes are data items ap-pearing on a screen or entered by the user. “MouseTrack(x1,y1)to(x2,y2)” means that the user highlights an areawith starting and ending coordinates (x1,y1) and (x2,y2)correspondingly.

The last three columns on the right of the table showthe results of the three phases of the task analysis process.At the end of this phase,URGENT has identified two user-specific variables (the user's login and password), a taskconstant (“a1” is the string entered in “Menu” to get the de-sired information about the claim), and two problem vari-ables (the customer's name and the number of his claim).It has also identified that the claim number can be obtainedfrom a standard position in the “Claim retrieval” screen.In a subsequent interactive phase, the legacy system useris presented with a graphical representation of this infor-mation and can edit the results in order to correct possible

errors.

The set of the different types of information manipu-lated in the analyzed task constitutes a small view to theoverall domain model of the system. In the example dis-cussed above, in the process of analyzing the “Claim Re-port Generation” task,URGENT has identified that “var6”and “var7” are two of the different types of informationthat the system uses. A second interactive phase enablesthe user to annotate these different pieces of informationwith a description of their semantics, such as, “var6 is-aCustomer-Name” and “var7 is-a Claim-Number”. In thismanner, the user creates a task-specific view of the sys-tem's domain model and a corresponding mini-databaseschema.

5.2. Task-based GUI design

Abstract User Interface Specification: After the anal-ysis of a task,URGENT specifies an abstract GUI, whichrequires the users to input their data items (user variablesand user input problem variables) only once, and buffersthem appropriately to deliver them to all screens that usethem. This GUI also retrieves from the legacy interfacethe “asked data items” and feeds them to the appropriatescreens. In this manner, if the user has developed a domainmodel, the GUI can also perform a “data-warehousing”task in the background of the interaction. So in the aboveexample, the users will enter their personal data and thecustomer name only once. The claim number will be au-tomatically retrieved from the standard area of the screenwhere it appears, and will be fed to the appropriate loca-tions of the subsequent screens that require it. This inter-action process is much simpler than the original one, wherethe users enter their personal data twice (once to each of thetwo subsystems), write down the claim number on paperand then enter this same number in three different screens

to get different elements of the final report3. At this point,for all data items to be manipulated, depending on theirtype, URGENT identifies a class of graphical interactionobjects appropriate for the data-entry action at hand [6].For example, for a tell action manipulating a date, appro-priate graphical objects might be a calendar, a combinationof three scrolling lists for year, month and day selection, asimple text entry box, etc.

Graphical User Interface Generation: The differenttypes of actions imply the need for different graphical ob-jects in the front end GUI. Finally, given a profile of theusers that perform the task, it generates a front-end GUIappropriate for users' requirements, based on standard in-terface design guidelines.

The final step in theURGENT process is the actual gen-eration of a GUI for implementing the user's task. Havingidentified a class of graphical objects appropriate for eachdata item,URGENT proceeds to develop adynamic HTMLGUI with graphical objects appropriate for the users forwhich this GUI is intended. To date, we have appliedthis process to simple report tasks, for which a simpleGUI generation process can generate sufficient new inter-faces. More complex tasks will require more elaborate ap-proaches to the problems of window organization and ob-ject layout.

6. Related and Further Research

The majority of work on reverse engineering has fo-cused on parsing the system code in various types of callgraphs [10], and based on them, on extracting higher-level,abstract structures [4]. While these methods were purelysyntactic originally, more recently they have incorporatedapplication-specific information such as a designer's modelof the system design [11] or a domain object model [2].To our knowledge, there has been no work that uses thetraces of thedynamic interaction between the user and thesystem.

Work on interface migration has followed similar strate-gies, although emphasizing the extraction of a high-levelrepresentation of the interface elements [7, 8]. And al-though developing task-specific interfaces is generally ac-cepted as a desirable practice for developing original inter-faces [14, 12, 15, 3], it hasn' t been applied to the migrationof interfaces across platforms. TheCELLEST method fortask modeling and interface migration relies on the concept

3This problem of interfaces requiring redundant information is notparticular to legacy systems only; rather it occurs when user interactionprocesses are distributed over multiple systems, developed independentlyof each other. For example, WWW users are increasingly faced with sim-ilar issues.

of task to collect related traces from which to extract thelogic and the elements of the information-exchange pro-cess between the user and the system. This approach local-izes the migration effort to different areas of the overall in-terface, and also enables the development of intuitive, user-friendly graphical interfaces tailored to simplifying the in-teraction of the user with the system in service of a specifictask.

Our own plans for further research include first theproblem of transition identification. Associated with that isthe problem of generating more semantic characterizationsfor the recognized system screens. Our current screen-recognition method relies heavily on characteristic featuresof the screens seen as images; we are now working ondefining features rich in semantic content, such as recog-nizing the application-specific keywords that most oftenappear on the system screens. Such features will enablethe generation of hypotheses about the screen behavior,which in turn, will support the transition recognition. Fi-nally, we are interested in feeding the results of the taskand domain modeling phase back to the interface mappingprocess, by using the task-specific data warehouses as re-sources for application-specific keywords that can be usedin the screen-recognition process.

7. Summary and Conclusions

Reverse engineering is the process of analyzing a sub-ject system to identify the system components and their in-terrelationships, and to create representations of the systemin another form or at a higher level of abstraction [17]. Thework we have described in this paper focuses clearly onthe latter goal. The basic idea underlying our work is that,for transaction-based systems, whose interface quite faith-fully represents the data model that they adopt and the tasksthat they accomplish, examination of the system-user inter-action traces can provide sufficient information forunder-standing the purposes of the system as they have evolvedand as they are currently perceived by its users.

In this paper, we described a process for mapping theinterface of a legacy system as a directed graph, basedon traces of system-user interaction. Then we describeda process for using traces of task-specific traversals of asub-graph of this overall graph to extract a model of theinformation-exchange process that occurs between the sys-tem and the user in service of this specific task. This ap-proach cannot support the maintenance of the system code,but it is useful for tasks such as interface migration and datawarehousing,

This approach exhibits two advantages over existingcode-based approaches:

� because it is interaction-driven and code-independent

it is geared towards extracting an information-exchange model of the underlying system, which canpotentially be closer to the requirements that the sys-tem was developed and evolved to deliver as opposedto low-level, possibly ad-hoc, implementation deci-sions;

� it directly supports the migration of the legacy inter-face to new task-specific GUIs.

The work reported in this paper is clearly work in progress,but we believe that the results of our initial experimenta-tion with the interface-mapping, task-modeling and inter-face migration processes are quite promising, and we con-tinue to develop and evaluate this process.

Acknowledgements

The authors would like to thank Roland Penner, BriceRiemenschneider and Satinder Sandhu for their assistancein the implementation. This work was supported by a gen-erous contribution from CEL corporation and a Collabora-tive Research and Development grant by NSERC 215451-98.

References

[1] 3270 Information Display System, Data Stream Pro-grammer's Reference, GA23-059-07, 8th Edition,IBM, June 1992.

[2] R. Clayton, S. Rugaber, & L. Wills: “Dowsing: ATool Framework for Domain-Oriented Browsing ofSoftware Artifacts” Automated Software EngineeringConference (ASE-98), October, 1998.

[3] M.R. Frank & J.D. Foley: “Model-based user inter-face design by example and by answering questions”,In Adjunct Proceedings of INTERCHI, ACM Confer-ence on Human Factors in Computing Systems, pages161-162, (Amsterdam, The Netherlands, April 24-29)1993.

[4] Wills, L., Automated Program Recognition by GraphParsing, MIT-AI-TR 1358, July 1992.

[5] L. Kong E. Stroulia & B. Matichuk: “Legacy Inter-face Migration: A Task-Centered Approach”, 8th In-ternational Conference on Human-Computer Interac-tion August 22-27, 1999 Munich, Germany.

[6] C. Lewis & J. Rieman: “Task-Centered User InterfaceDesign”, http://www.acm.org/ perlman/uidesign.html,1993.

[7] E. Merlo, P.Y. Gagn, J.F. Girard, K. Kontogiannis, L.J.Hendren, P. Panangaden & R. De Mori: “Reverse en-gineering and reengineering of user interfaces”, IEEESoftware, 12(1), 64-73, 1995.

[8] M. Moore: “Representation Issues for Reengineer-ing Interactive Systems”, ACM Computing Surveys,28(4), 199-es, 1996.

[9] M. Moore & S. Rugaber: “Issues in User Interface Mi-gration”,Proceedings of the Third Software Engineer-ing Research Forum, Orlando, FL, Nov 10, 1993.

[10] H.A. Mueller, M.A. Orgun, S.R. Tilley, & J.S. Uhl:“A reverse engineering approach to subsystem struc-ture identification”, Journal of Software Maintenance:Research and Practice, 5(4), pages 181-204, December1993.

[11] G.C. Murphy, D. Notkin, K. Sullivan: “Software re-flexion models: bridging the gap between source andhigh-level models”, ACM SIGSOFT Software Engi-neering Notes Vol. 20, No. 4 (Oct. 1995), Pages 18-28.

[12] M. Sanz, & E.J. Gomez: “Task Model for Graph-ical User Interface Development”, Grupo de Bioin-genieriay Telemedicina, Universidad Politecnica deMadrid, Technical Report gbt-hf-95-1, 1995.

[13] S.N. Srihari, S.W. Lam, V. Govindaraju, R.K. Sri-hari & J.J. Hull: “Document Image Understanding”,Tech. Report CEDAR-TR-92-1, Center of Excellencefor Document Analysis, State University of New Yorkat Buffalo. May 1992.

[14] A. Vanniamparampil, B. Shneiderman, C. Plaisant& A. Rose: “User interface reengineering: A diag-nostic approach”, University of Maryland, Departmentof Computer Science, Technical Report CS-TR-767,1995.

[15] S. Wilson, & P. Johnson: “Empowering Users ina Task-Based Approach to Design”, Proceedings ofDIS' 95, Symposium on Designing Interactive Sys-tems, Ann Arbor, Michigan, August 23-25, pp.25-31,ACM Press, 1995.

[16] http://www.celcorp.com

[17] http://www.tcse.org/revengr/taxonomy.html

[18] LOCIS IP address: locis.loc.gov