Computer Methods and Programs in Biomedicine 22 (1986) 199-207 199 Elsevier

CPB 00763

Portable file management system in FORTRAN. II. The input/output routine for free-format text

Masah iko O k a d a and M i h o k o O k ad a

I Division of Medical Informatics, Niigata University Hospital, 1 Asahimachi, Niigata City 951 and 2 Department of General Education, Niigata University, 2-Nocho, Ikarashi, Niigata City 950-21, Japan

A software tool for inputing and outputing patient data ( 1 / O routine) has been developed. Since this I / O routine is programmed exclusively in FORTRAN77, it will make a powerful tool for constructing a portable database system. Basically the routine manipulates an ASCII-coded text string that consists of lines demarcated by the CR code (13) and is terminated by the null code (0). The editing commands are preceded by one of the following ASCII characters: @, !, ], [, *, and _ , and all the strings with an initial character other than these are interpreted as data to be inserted into the text. Since the routine uses two F O R T R A N tools already reported, i.e. the subroutines to manipulate key files and the subroutines to manage variable length records, character strings can be stored without any restrictions in format or in size, and can be retrieved either sequentially or in an indexed manner.

Management system Data management Portable database system

1. Introduction

Basically, there are two types of preferable ap- proach to the organization of large-scale data- bases, one of which is to depend on an informa- tion center with multiple data-entry terminals in remote locations [1,2], and the other is to de- centralize data files over multiple sites using com- mon or portable software [3,4]. There are many restrictions that impede linkage of clinical records which have already been developed in different computer systems [5,6], although some efforts have been made towards generalizing data-description languages [7] and data-exchange standards [8] to enhance program and data transportability.

As remarked by Kuhn et al. in their follow-up study [9]: 'While high level languages cannot com- pletely mask differences in the underlying operat- ing system and hardware, the use of high level languages would facilitate system transfer.' Some

comparative studies [9-11] have revealed FOR- TRAN, MUMPS, P L / I , COBOL and BASIC as the major high-level languages used in clinical researches. Previous discussion on system porta- bility, however, has been restricted to MUMPS, which several authors insist is highly transferable [3,4,12,13].

We have been studying a FORTRAN-based database system, FLXFL, since 1979 [14,15]. Using this system, more than 100 000 records of a gastric cancer screening test [16] are now managed. Through this seven-year experience on clinical data management with FLXFL, we are convinced that FORTRAN is a not inappropriate language for large-scale database management. However, even though the system was principally written in FORTRAN77, it was necessary to modify the program for system transfer, because some input /ou tput procedures, such as those for CRT terminals, were machine dependent.

0169-2607/86/$03.50 © 1986 Elsevier Science Publishers B.V. (Biomedical Division)

200

In this paper, we present a subroutine to input and output patient data (we call this subroutine the I / O routine). The I / O routine is written ex- clusively in the standard instruction set of ANSI- FORTRAN77, and neither assembly language nor machine-dependent routines are used. In particu- lar, since input /output procedures of the text editor are processed by Read and Write (or Accept and Type) statements, any types of conventional CRT-and-keyboard terminals may be adapted to this I / O routine. Although there are probably hundreds of different text editors in use today [17,18], no portable editors exist. Using the routine, it is possible to edit and store patient data without any restrictions in format or in size, and to retri- eve them sequentially or in an indexed manner. Non-restricted free size formats allow the han- dling of any types of data, including numerical, coded, and free-text [19], and are suitable to the management of time-oriented data. In this article we will describe the details of this routine and discuss the portability [20] of the system.

2. Background

The I / O routine uses two software tools which we recently developed for programming highly porta- ble file management systems. One of them is the subroutine set to manipulate key files applying the B-tree, which is a form of index structure [11]. In order to create a B-tree of order m, a file of fixed length records is prepared. Each record, which is considered to be a node, contains three arrays for m key values, m indexed record addresses, and m + 1 pointers, and is linked with pointers in a tree structure. The program is composed of six subroutines, (Initialize, Open, Close, Insert, Search and Delete), and consists of 159 FORTRAN statements in total. It is able to handle up to nine keys simultaneously, and is designed to permit the occurrence of duplicate key values.

The other tool is the subroutine set to save records of variable length [21]. In the method employed a record, which is assumed to be a character string such as an ASCII-coded text terminated by the NULL code (0), is compressed and divided into 64-byte-length blocks. One block

consists of a data field and a pointer field, and the blocks comprising a record are chained with pointers forwardly. Consequently the tailing part of the last block is not always filled with data. When an existing record has been edited and increased in size, non-contiguous storage space is used to save the blocks. These data are stored in a file called a data file. A head pointer of each record is sequentially saved on a separate file (the pointer file). The data compression is performed as follows; if the same character code appears more than twice in succession, we count the num- ber of the repetitions and save it with the initial two characters. The program is composed of six subroutines (Initialize, Open, Close, Insert, Fetch and Delete) and consists of a total of 185 FOR- TRAN statements.

3. System description

3.1. Data structure

A text, which is stored in the variable-length-re- cord file, is assumed to be a string of ASCII characters. The terminator of a text is the N U L L code (0) and the delimiter which separates each line is the RETURN code (13), as shown in Fig. 1. In this figure, the string CR CR CR CR stands for three lines with no visible characters. The last two codes of a text are always CR and NULL, as explained below. On displaying a text, whenever the output routine encounters the CR code, it returns the carriage and advances one line. A displayed image of the example in Fig. 1 is shown in Fig. 2.

3.2. File preparation

The I / O routine possesses 12 functions: initializ- ing the directory, displaying the directory, initial- izing a file, adding a new record, modifying a record already inserted, deleting a record, initializ- ing a key file, renewing key files, retrieving a record in the sequential manner, retrieving in the indexed manner, displaying the command menu, and exiting. Each function is activated by striking

continued

! cursor

Fig. 1. Structure of text. The terminator of text is the NULL code and the delimiter which separates each line is the RE- TURN code. The arrow shows the cursor position.

a key corresponding to the item number displayed on the Command Menu.

After running, the system initially outputs the following Command Menu on the screen:

1. Edit

2. Directory

3. Initialize

4. Key file (1)

5. Help

6. Exit

In order to prepare a new file, we choose Initialize by striking 3 on the keyboard. The routine then outputs the following message:

rNy, 1 M. OKADA

i 2 NIIGATA CITY 3 4 5 6 NORMAL 7m

/

/ J

Fig. 2. Displayed image of the example shown in Fig. 1.

\

201

Data file name? (2)

The file name must be specified in less than 21 characters according to the naming rule of the operating system used. Then the following is dis- played:

Pointer file name? (3)

As described above, the system always uses a pair of files: the data file and the pointer file. On receiving a reply, the system returns to the initial state and displays menu (1).

3.3. Text editor

By choosing Edit from menu (1), the system out- puts the following:

I. Maintenance

2. Apend

3. Modify (4)

4. Retrieve

5. Delete

6. Exit

\ /

1 •

/ \

Fig. 3. The frame initially displayed by the text editor. Only the

first line number and the cursor are shown.

202

122 2 3 1124 125 126 127 128 129 130 131 132 133

190/110 37 235 660 127 127 49 1985.09.20 var iola,measles,var icel la dyspnea,cyanosis,edema Appetite has been good Nutr i t ion is normal Heart is normal

134 He has an a l le rg ic diathesis =

/

/

\

Fig. 4. When the Modify command is issued, the content of the designated record is displayed on the screen by scrolling up.

In any menu, Maintenance always brings the sys- tem back to menu (1) and Exit terminates the editor. The editor is activated by Apend or Mod- ify, where both commands are equivalent except that the former initially displays only the first line number on the screen (Fig. 3) and the latter asks for the record number to be edited as follows:

Record number? (5)

Having chosen the Modify command and answered this question, the whole contents of the designated record are displayed on the screen by scrolling up (Fig. 4). The screen in Fig. 4 shows the last 14 lines of an example text. The editor always dis- plays the line numbers sequentially on the left-most three columns of the screen. An input string to the editor is interpreted as a command if it is started with one of the following special characters (Table 1):

@ .' ! I * - (6)

If it is started with another printable character then this is simply inserted into the text being edited. Editor commands (including insertions), which must be less than 80 characters, are not executed until the R E T U R N key is depressed.

There are four commands to move the cursor. The command:

@n (RETURN) (7)

moves the cursor to the terminal CR of the n th line. The example in Fig. 1 shows the cursor position after executing @6. When n is larger than the last line number existing, CRs are automati- cally inserted for the excess, and the cursor skips to the line: this function makes it easy to input records with many empty fields.

The command:

!aaaaa(RETURN) (8)

searches for the string 'aaaaa ' and moves the cursor to the character next to the string first found. The search for the same string may be repeated by simply entering;

! (RETURN) . (9)

If the string is exhausted, the search is restarted from the beginning of the text in a cyclical fash- ion. An example in Fig. 5 shows the cursor posi- tion after executing the two commands !normal and ! (assuming that there is no such a string before the 119th line). Strings to be searched for cannot contain the space character at the end. To move the cursor to the last position of the current text, the command ! with a succeeding character

\ /

l lg 340.0 120 152.9 121 190/110 122 37 123 235 124 660 125 127 126 127 127 49 128 Ig85.0g.20 129 variola,measles,varicella 130 dyspnea,cyanosis,edema 131 Appetite has been good 132 Nutr i t ion is normal ~ 33 Heart is normall

.X / \

Fig. 5. The command !normal catches the string 'normal' which first appears after the current cursor position. The command ! catches the same string when it next appears.

which might never appear in the text, e.g. !!(RE- T U R N ) , is preferable. Depressing only the RE- T U R N key:

(RETURN) (I0)

advances the cursor one line. Deletions are done on the basis of a character

or of a text. The com m a nd ] deletes a character preceding the cursor but never involves the C R code. On the contrary, the command I deletes any characters in the current text including the CR. Two or more I (or 1) can be entered at once as follows:

III!11 (RETURN). (11)

Fig. 6 shows the status after executing c o m m a n d (11) on the example of Fig. 5. In order to delete all the characters in the text, the c o m m a n d *, which simply moves the N U L L code to the first position of the text, is used.

A string to be inserted cannot contain a space at the first location, and tailing blanks are ignored. After the execution of an insertion, all the char- acters preceding the cursor in the current line are re-displayed, so the cursor will appear on the same posit ion as before. For example, the following insertion to the text in Fig. 6:

\ /

If9 340.0 120 152.9 12l 190/If0 122 37 123 235 124 66O 125 127 126 127 127 49 128 1985.09.20 129 variola,measles,varicel la 130 dyspnea,cyanosis,edema 131 Appetite has been good 132 Nutr i t ion is normal 133 Heart is • x.

\

203

enlarged(RETURN) (12)

results in Fig. 7. If and only if a string is inserted on the last position of the last line, the C R is automatical ly inserted into the following position, and the cursor moves to the next line, which is newly appended, so that the last two codes of text are always CR and N U L L , as described before (Fig. 1). When only the R E T U R N key is de- pressed, no insertion occurs (if the cursor is on the last character position of the text, the C R code is inserted) and the cursor moves to the last char- acter (which is always CR) of the next line. C R

^

can be inserted into the text by placing , which is internally converted to the CR code, as follows:

enlarged ^There is a systolic th r i l l (RETURN).

(13)

The screen before and after the execution of (13) is given in Figs. 6 and 8, respectively. After the insertion, the subsequent line numbers are incre- mented by one. The allowable characters for inser- tion, are therefore the letters A to Z and a to z, the numerals 0 to 9, all the printable special char- acters, and the space.

To exit the editor, enter:

_ ( R E T U R N ) (14)

the routine then returns to menu (1).

\ /

340.0 120 152.g 12l Ig0 / l l0 122 37 123 235 124 660 125 127 126 127 127 49 128 1985.0g.20 ]29 var iola,measles,var icel la 130 dyspnea,cyanosis,edema 131 Appetite has been good 132 Nutr i t ion is normal 133 Heart is enlargedB

X / \

Fig. 6. The string 'normal' has been deleted by the command Fig. 7. The word 'enlarged' has been inserted. Note the cursor HlllJ(RETURN). At this time, the cursor never advances, position.

204

%

!

121 122 123 124 125 126 127 128 129 130 13l 132 133 134 x.

/

/

152.9 190/110 37 235 660 127 127 49 Ig85.0g.20 var iola,measles,var icel la dyspnea,cyanosis,edema Appetite has been good Nutr i t ion is normal Heart is enlarged There is a systo l ic t h r i l l |

.X \

Fig. 8. A new field which occupies one line has been inserted immediately after the 133th line. The subsequent line numbers are automatically incremented by one.

3.4. Special functions of the first record

The first record of a data file presents some spe- cial functions. This record can be used not only for the usual storage of patient data using the editor commands, but also for the purpose of formatting texts and manipulating key files. The letter d or k written in the first position of each line causes the following actions. The letter d of the ith line makes the string written in the line to be the default values for the following data input: if only the R E T U R N is depressed for the ith line of the new record, this default value is automati- cally inserted into the line. The letter k is for specifying a key field and a key file. The format for setting a default value is

d aaaaaa (RETURN) , (15)

where aaaaaa is a default value. Key specification is done by:

k nn f f f f f f f f (RETURN), (16)

where nn is the number of characters (less than 20) that constitute the key field, and 'ffffffff ' is the name of the key file; the naming rule of the file is the same as that of (2) and (3). An example of (16) is;

k 08 D L I : T E S T • KEY(RETURN) . (17)

Two or more of (15) and (16) can simultaneously be specified in the record, and these can be added or deleted any time during editing.

3.5. Key file

Key files can be initialized or renewed by choos- ing Key file from menu (1). The following menu is then displayed.

1. Maintenance

2. Insert (18)

3. Initialize

4. Exit

The option Initialize creates a new key file. The routine then asks for the key to be identified:

Which file? (19)

and

Which item? (20)

where ' i tem' is equivalent to 'line'. The key file is then created according to the specification in the form of (16). If there is no specification in the designated line of the first record, the message:

Not a key field (21)

is displayed. In either case, the routine returns to menu (18) afterward.

Insert in menu (18) is for inserting key values into the file thus created. When Insert is selected, the questions:

Which file? (22)

Initial record number? (23)

and

Final record number? (24)

are asked in series. After answering these three questions, all the key files that are specified in the

first record are updated, i.e. the key values of the records in the specified range are inserted into each key file. If a file specification is deleted from the first record, that key file will not be involved in the subsequent renewal. After the insertion menu (18) is again displayed.

3. 6. Retr ieval

In menu (4), choosing Retrieve offers two func- tions, sequential and indexed searches' of records, in the form of the following menu:

1. Edit

2. Sequential (25)

3. Indexed

4. Exit

The option Sequential retrieves all the records in the range specified by the user (by answering again questions (23) and (24)), and outputs the contents of them sequentially on the screen.

The option Indexed performs retrieval of re- cords indexed by the key that is specified by the user answering the questions:

Which item? (26)

and

Key value? (27)

The first record thus found in output on the screen, and the message:

1. Continue the search (28)

2. Return to the menu

is displayed. The former option continues the search to get the next record that meets the condi- tion, and displays it. When the records to be searched are exhausted, the message:

Not found (29)

is output. In either case, i.e. option 2 is chosen from menu (28) or when output (29) is obtained, menu (25) will again be displayed.

- DATA FILE - POINTER FILE

TEST.DAT TEST.PNT WORK DAT WORK.PNT

3 CANCER.DAT CANCER.PNT 4 SCREEN.DAT SCREEN.PNT 5

205

/

Fig. 9. The option Directory provides a listing of the files.

\

3. 7. Deletion

When the function Delete is selected in menu (4), the following message is displayed;

Delete! are you sure? (30)

The reply by the letter y to this question urges the routine to proceed with the delete operation, and other replies cancel the command. Having answered the question that asks for the record number to be deleted, the contents of the record are displayed. The same message as that of (30) is displayed again, and the record is deleted finally by depressing the y key. The empty areas in the file thus generated are squeezed by the variable-

TABLE 1

The editor commands

Command Function

n

!aaaaa !

I I

move the cursor to the line numbered n move the cursor to the string aaaaa move the cursor to the string previously defined move the cursor to the end of the current text delete one character excluding CR delete one character including CR delete all the characters in the current text quit the editor

206

length-record routine according to the algorithm described in [21].

Files, once created, cannot be deleted by the I / O routine. The only way to purge files is to use the command for deletion in your operating sys- tem.

3.8. Other facifities

The option Directory in menu (1) provides a list- ing of files already created (Fig. 9), and Help provides a listing of the editor commands in the same style as that of Table 1.

4. Portability

One of the factors that hinder the transport of software to different computer systems is that the different systems employ quite different naming rules of files. Through the I / O operations de- scribed so far, the user can assign a file name by answering the question asked by the routine. The only exception is the directory file. The directory file is created by typing '999,' which is not di- rected anywhere in the menus, when menu (1) is first displayed. If this operation is executed again, the directory file is initialized. In the proposed program, the name of the directory file is defined in the first and second lines as;

DATA V L R D I R / ' V L R D I R . D A T ' / CHARACTER* 21 VLRDIR

(31)

The first and probably the only thing to do to implement the routine is to change the literal in this DATA statement.

The second factor to be considered is the facili- ties of READ, WRITE, and FORMAT state- ments, which are slightly different from system to system. In our routine, the following are expected. (1) The statements:

BYTE BUF(80)

READ(5,100) BUF 100 FORMAT(80A1) (32)

continue to input characters until RETURN is

entered or the number of characters reaches 80. The remainder of the array BUF is filled with the space code. When only RETURN is entered, all the elements of BUF are set to the space code. (2) As for the output, the statements:

WRITE(5,200) BUF(I) 200 FORMAT(1H + ,1A1,$) (33)

write only one character, suppressing line feed before printing and carriage return after printing. (3) Therefore the output by (33) immediately after the execution of the READ statement is over- printed on the same line, so the screen will look like Fig. 7.

The third problem is the specifications for di- rect-access-file attributes. For example, the state- ment:

OPEN(UNIT = 3,FILE = VLRDIR,

ACCESS = 'DIRECT' ,STATUS =

' U N K N O W N ' , RECL = 11) (34)

is used in the I / O routine to create the directory file. By this statement, an unformatted file is opened whose logical unit number is 3. The name is assigned by the character constant VLRDIR, the allowable access method is direct, and the record length is 11 X 4 bytes. For the file thus created, the statement:

WRITE(3 'K) n (35)

writes the contents of n into the Kth record, where the value of K never changes before or even after the execution.

The the files used in the routine are direct- access-files provided by FORTRAN, which have the same physical structure as a sequential file: the simplest structure. Therefore, the transfer of the program and data files will not cause any serious problems.

5. Application

The routine is tentatively used for the manage- ment of the data of heart disease screening for

children, where 16 000 records are stored annually. Each record consists of 26 data items: date of the examination, ID number of examinee, school code, past history, family history, symptoms, chest X-ray findings, ECG findings and PCG findings. Only date and ID number are stored for the examinees whose test results are normal, hence the record size is quite different from examinee to examinee. These data are being managed quickly and flexi- bly by this system.

6. Hardware and software specifications

The proposed program was tested on a PDP 11/44, a 16-bit-word machine with a RA80 disc storage unit (121 Mbyte), and several VT100 character display terminals, under the RSX11/M operating system. The program, however, does not depend on any characteristics of the particular computer systems.

7. Mode of availability

The source list a n d / o r magnetic tape copy (1600 BPI) of the I / O routine, accompanied by two subroutine sets described in Section 2, is available from the authors.

References

[1] J.M. Ullom and G.S. Cohen, Networking clinical experi- ence, in: Proceedings of Computer Networks COMPCOM '82, pp. 1-5 (IEEE, New York NY, 1982).

[2] M.J. Gillespie, L.D. Fisher, K.B. Davis, et al., Distributed data entry in a large collaborative medical study (CASS): seven year experience, in: Proceedings of Computer Net- works COMPCON '82, pp. 627-634 (IEEE, New York NY, 1982).

[3] T. Munnecke, Software portability considerations for mul- tiple applications over multiple sites, in: Proceedings of the Fifth Annual Symposium on Computer Applications in Medical Care, pp. 38-42 (IEEE, New York NY, 1981).

[4] C. King, R.M. Strong and L. Goldstein, Comparing data management systems in clinical research: a survey, in:

Proceedings of the Fourth Annual Symposium on Com- puter Applications in Medical Care, pp. 1-30 (IEEE, New York, NY, 1980).

207

[5] K. Sauter, P.L. Reichertz, W. Weingarten, et al., A system to support high-level data description and manipulation of an operational data-base system, Med. Inform. 1 (1976) 15-26.

[6] G.W. Beebe, Record linkage systems - - Canada vs the United States, Am. J. Public Health 70 (1980) 1246-1248.

[7] Y.K. Oon and N. Carson, The PORTA language: a porta- ble medical record system, Aust. Comput. J. 17 (1985) 88-90.

[8] C. McDonald, G. Wiederhold, D.W. Simborg, et al., A discussion of the draft proposal for data exchange stan- dards for clinical laboratory results, in: Proceedings of 1984 Symposium on Computer Applications in Medical Care, pp. 406-413 (IEEE, New York, 1984).

[9] I.M. Kuhn, G. Wiederhold, J.E. Rodnick, et al., Auto- mated ambulatory medical record systems in the U.S., Report No. STAN-CS-82-928, pp. 1-66 (Department of Computer Science, Stanford University, Stanford CA, 1982).

[10] C.H. King, R.M. Strong and L.P. Goldstein, Distribut- ing database management for patient-care research: MEDUS/A, in: Proceedings of the First IEEE Computer Society International Conference on Medical Computer Science/Computational Medicine, pp. 1-15 (IEEE, New York NY, 1982).

[11] M. Okada, M. Okada and M. Yakata, Portable file management system in FORTRAN, Comput. Prog. Bio- med. 19 (1984) 47-51.

[12] R.A. Greenes, A.N. Pappalardo, C.W. Marble, et al., Design and implementation of a clinical data management system, Comput. Biomed. Res. 2 (1969) 469-485.

[13] G.O. Barnet, MUMPS: an evolutionary commentary, Comput. Biomed. Res. 14 (1981) 112-118.

[14] M. Okada and M. Okada, Conversational file processing system FLXFL (FLeXible FiLe) for nonprogrammer, Comput. Biomed. Res. 12 (1979) 335-349.

[15] M. Okada and M. Okada, The pursuit of a total data processing system free of language and programming, Comput. Biomed. Res. 17 (1984) 193-207.

[16] M. Okada and M. Okada, An evaluation of data items for filling system of a gastric cancer screening project, in press.

[17] T.L. Roberts and T.P. Moran, The evaluation of text editors: methodology and empirical results, Commun. ACM 26 (1983) 265-283.

[18] J. Gutknecht, Concepts of the text editor Lara, Commun. ACM 28 (1985) 942-960.

[19] N. Sager and R. Wong, Developing a database from free-text clinical data, J. Clin. Comput. 5/6 (1983) 184-194.

[20] P.J. Jalics and T.S. Heines, Transporting a portable oper- ating system: Unix to an IBM minicomputer, Commun. ACM 26 (1983) 1066-1072.

[21] M. Okada and M. Okada, An algorithm to manage varia- ble length records for highly portable clinical database systems, Comput. Biomed. Res. (1986) in press.