IMPLEMENTING AN ELECTRONIC RECORDS

MANAGEMENT PROGRAM

Philip C. Bantin Indiana University Archivist bantin@indiana.edu IU Electronic Records Program

Website: http://www.indiana.edu/~libarch/ER/NHPRC-2/index.html

OUTLINE OF PRESENTATION

1) Requirements for an Electronic Records Management System (ERMS)

2) Why On-Line Transaction Processing (OLTP) Systems, Data Warehouses, and Electronic Document Management Systems (EDMS) Do NOT Meet These Requirements

OUTLINE OF PRESENTATION

3) Capturing Records – Strategies ** Business Process Engine –

Example: European Registry System

** Workflow or Routing Engine – Example: IU Workflow Engine

** Records Management Applications (RMA) – Example: TRIM

4) Capturing Records: Metadata Specifications

Information Systems Systems Development Lifecycle

System concept: purpose, goals, scope Analysis: user/functional requirements Design

data design: what information? software design: processed how? interface design: user interaction?

Coding and testing: execute & evaluate Key issue: Systems do (only) what

they’re designed to – purpose, goals, scope, requirements.

Data and Information Systems

Transaction Processing Data-oriented Automate basic business

processes Decision Support

MIS/EIS Data warehouse

Electronic Document Management Systems (EDMS)

Transaction Processing Systems (TPS) Employing DBMS Software

The most basic business system and the heart of most organizations

TPS is a computerized system that performs and records the daily routine transactions necessary to the conduct of business

Primary goal is to automate computing intensive business transactions, such as those undertaken in the financial and human resource functional areas

TRANSACTION PROCESSING SYSTEMS

Inputs=transaction data like financial information, human resource data

Processing=sorting, listing, updating, merging

Outputs=bills, paychecks, orders, detailed lists and reports

Transaction Processing Data Representation and Storage

Sequential files ordered set of structurally similar records

Indexed files 2 part structure: data + index

Databases multiple entities (tables) linked by key-fields

minimal redundancy

Data Systems and Record Systems

How are they Similar? They both…

capture and store data organize digital data enforce standards for representation

of data protect data from destruction

(accidental or intentional). make data available for people to

use (search, browse, retrieve, etc.)

But are TPS Good Recordkeeping

Systems? Transaction Processing

Primary Goal: to automate some activity or business process.

Transaction Processing and Records Recordkeeping may be the business process …or records may be the byproduct of the

business process; but recordkeeping is not the primary purpose of the system

…or records may be difficult or impossible to locate or retrieve …if they exist there at all.

Transaction Processing Record-oriented business process

General Ledger keep record of debits and credits cumulative history of transactions:

account, object, amount, date… Academic Record (Transcript)

keep record of courses, grades, degrees

cumulative history of academic career: course department & title,

instructor, grade semesters and dates of enrollment degree, school, major, honors, date

Transaction Processing Records as byproduct

Hotel Reservation retrieve room vacancy information collect guest information assign guest to room record guest information & reservation dates

Employee Payroll combine pay rates and hours worked produce paychecks record annual totals of earnings,

withholdings, benefits But managing complete records of all

business transactions over the life cycle is not the primary objective of the system

Transaction Processing Systems and

Record Systems

Why aren't TPS reliable Recordkeeping Systems? They capture incomplete information.

Records are made up of content, context, and structure. Many Information Systems capture only content, and not always completely.

They keep only current information. Many Information Systems store only "current values" for key bits of information and they do a bad job of tracking the history of changes to these values.

Transaction Processing Systems and

Record Systems

Why aren't TPS reliable Recordkeeping Systems? They scatter information. To avoid

redundancy, Information Systems store each bit of information just once, often in separate tables or databases, and they rely on computer software to re-connect the bits of information when needed. Related pieces of information become disconnected over time, or may be preserved or discarded according to different timetables.

Transaction Processing Systems and

Record Systems

However…in many cases TPS ARE (de facto) Record Systems. An institution or agency may be using a TPS

to conduct its business, and relying on the databases of that system to keep a record of key activities.

It may not be a really good Record System, but that Data System IS the institution's (agency's) primary system for keeping records.

Key issue: Is it possible to make an adequate Record System from a Transaction Processing System?

DECISION SUPPORT SYSTEMS

A computer-based information system that provides a flexible tool for analysis and help managers in making strategic decisions

Uses data and information to produce flexible, on-demand reports or information, and assistance in decisions about unstructured problems

By means of models, DSS can provide valid representations of real world systems

DATA WAREHOUSE The data warehouse concept is about

unbundling the two environments. In one environment – TPS - the business automates its processes on many different on-line transaction systems in the most effective and expedient manner possible. The data from these many and varied systems is then used to populate a database comprising all the data necessary to support decision making in a separate data warehouse environment.

DATA WAREHOUSE Unlike operational databases that

are set up to handle transactions and that are kept current as of the last transaction or update, data warehouses are analytical, subject oriented and are structured to aggregate transactions as a snapshot in time.

DATA WAREHOUSES - PRIMARY CHARACTERISTICS

Separate from the Operational systems and populated by data from these systems

Available entirely for the task of making information available to users

Time-stamped and associated with defined periods of time

Subject Oriented as defined by the customer

Accessible to users who have limited knowledge of computer systems

Data Warehouses as Recordkeeping Systems Data Warehouses are NOT ERMS Do not meet many of the

requirements 1) Do not routinely capture records

– Evidence of Business Transactions – More concerned about transforming data into information

2) Do not manage information or records over the entire life cycle

EDMS as Recordkeeping Systems Many EDMS functions overlap with

ERMS EDMS functionality typically includes:

indexing of documents, storage management, version control, integration with desktop applications, and retrieval tools to access the documents

But they are not fully functioning ERMS

EDMS As A Recordkeeping System

EDMS: Primary purpose is to support day-to-day use of documents for ongoing business

ERMS: Primary purpose is to provide a secure repository for authentic and reliable business records

EDMS AS RECORDKEEPING SYSTEMS

WHAT IS A DOCUMENT? “A grouping of formatted information

objects regardless of medium or form that can be accessed and used by a person.”

“A document is a container which brings together information from a variety of sources, in a number of formats, around a specific topic, to meet the needs of a particular individual.”

A document is discrete and identifiable - Structural unit of text like a report or letter or minutes of meetings

EDMS AS RECORDKEEPING SYSTEM

WHAT IS A RECORD? A specific type of information

produced by a business event Evidence of business transactions Evidence is metadata documenting

the context of creation, and the content and structure of the record

Forms the basis for Recordkeeping Systems

EDMS As A Recordkeeping System

EDMS allows documents to be modified and exist in several versions

ERMS prevents records from being modified

EDMS As A Recordkeeping System

EDMS may allow documents to be deleted by owner of data

ERMS prevents records from being deleted except in certain strictly controlled circumstances

EDMS As A Recordkeeping System

EDMS may include some retention controls

ERMS must include rigorous retention controls

EDMS As A Recordkeeping System

EDMS may include a document classification scheme

ERMS must include a robust record classification scheme

EDMS As A Recordkeeping System

Most EDMS do not meet many of the requirements of a EDMS

1) Do not routinely capture records – Evidence of Business Transactions

2) Do not manage information or records over the entire life cycle

Strategies for Capturing Record

Content and Record Metadata

Overall Goals of Record and Metadata Capture

System officially “captures” records for the University

Capture involves process of: 1) Registering a record 2) Deciding which class it

should be classified to 3) Adding further metadata to it 4) Storing it in the ERMS.

STRATEGIES FOR CAPTURING RECORDS

1) Applications that include BUSINESS PROCESS ENGINES

2) Applications that include WORKFLOW ENGINES

3) RECORDS MANAGEMENT APPLICATIONS (RMA)

Business Process Models

Primary Advantage: Models clearly identify all activities

within a process and consequently identify all records and the relationships between records and processes

It is a complete representation of the business process and of the various inputs and outputs generated by the process

Business Process Models

Primary Disadvantage These models are not often

designed into the system - automated business process engines are not the norm in North America

Good Examples of automated business process engines are the European Registry Systems

Registry System in Finland

All recordkeeping functions in a Finnish government agency are supervised by a recordkeeping schedule

Based on a hierarchical classification of agency functions, activities and business processes which serves the registry system filing appraisal

Registry System - Operational Procedures

Registration is based upon a business process e.g, filling an archivist’s position in the

university archives This business process gets a registry

number that is based on the functional classification schema uniquely identifies the business process in

question links the business process to the function it

serves, in this example personnel management

Registry System - Operational Procedures

Each business process is registered step by step regardless of the boundaries of the organizational units that participate in the process 

Records created by each step are filed in a document management system and are linked to registry entries

Example of Recordkeeping in Registry System - Filling an archivist’s position in the University Archives

Date Transaction Agent

2001-04-24 an announcement of the vacant position sent Office A Rto a local newspaper

2001-05-10 an application sent by Ms. X Office A R2001-05-10 an application sent by Ms. Y Office A R2001-05-12 an application sent by Mr. W Office A R2001-05-24 applications sent for review to the archives Office A2001-06-24 statement of the applicants sent to Office B Archives R

after interviews2001-06-30 a request for additional information from Office B R

the archives2001-07-10 a revision to a previous statement to Office B Archives R2001-07-25 a decision to appoint Ms. Y as an archivist Office B R2001-07-26 a letter to Ms. Y informing her that she has Archives R

been appointed

Advantages of the Registry System For records and archives

management the registry links records to business processes and functions

that create them other records created by the same

business process and function Provide contextual information and

ensure the integrity of electronic records

Conceptual Design – Workflow

Workflow is "the automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant [human or machine] to another for action, according to a set of procedural rules.”

http://www.e-workflow.org/

“Starting from creation and ingestion, we should integrate the workflow process with the preservation process: appraisal, verification, maintenance and, eventually, retirement.”

Su-Shing Chen “The Paradox of Digital Preservation”Computer (IEEE Computer Society), March 2001

WORKFLOW MODELS Advantages: 1) Commonly used 2) Coming back into fashion with an emphasis on life cycle management

3) Often Automated

WORKFLOW MODELS Disadvantages 1) Primarily a Routing mechanism

and thus: a) not all records that are created go

through workflow process b) routing process may not identify

all activities, all inputs and outputs within a given process; consequently some records within the process may not be identified and captured

UIS EDEN Workflow Engine

Overview of workflow engine for IU’s OneStart portal.

OneStartCustom

ized

Personalized

Adaptable

Desktop

Application Delivered

HR

MS

SIS

FIS

IUIE

Other

Other Content

EDEN

Channels

Services

Workflow

Record Keeping

Security

Users Application

Services

Applications

User Interface

Infrastructure

OneStart & EDEN Component-Based Development

Goals of EWE Service to enterprise applications

that routes electronic transactions to individuals or systems for work, approval or notification.

Audit trail of all routing and actions taken on electronic transactions.

Integration with IU’s portal.

Technical Architecture Component Based – EJB’s

Application interface Application post-processors Route Modules

Documents processed in XML format Minimal content is EWE route control data EWE can be used as the storage for pending

transactions. EWE can enable versioning of information.

Web services – applications required to register a web service for each document type.

Document creation

Request route ID for doc

Application using EWE

Workflow engine

Applications send XML version of electronic document to EWE. (Document must be of a registered type.)

Route Document

RouteDoc( XML contents of doc )

Application using EWE

Workflow engine

EWE submits XML to route modules

Discover Action Requests

Workflow engine

1) XML for document

2) Action Requests

3) Add action requests to

document route

Routing Modules

Route modules scan XML for specific route controls and match document information to action request rules.

Route modules return list of action requests

Activate Next Action Request Document’s action requests are

activated by:1. Route type priority as determined by

document type route template2. Request priority as determined by the route

rule3. Order in which request was associated with

the document. The activated requests are put on the

appropriate action lists of the persons or processes.

Review Action List Person reviews

action list in the portal.

Person is able to see the routing log and other header information.

Open document Action list refers user

to original application to see any business information in the document.

Application consults with engine to determine what elements of the document to show the user, if any.

Approve Document User acts upon

the action request.

Application business rules are applied to control actions taken on the document.

Route Document again

RouteDoc( updated XML contents of doc and action taken by user )

Application using EWE

Workflow engine

Post-process document Once the engine has determined that

all routing is complete, it then notifies the application post-processor.

This is an EJB for the application that does appropriate work with the document such as changing pending status, updating calculations, generate new documents, etc.

ADDITIONAL FEATURES

Add Recordkeeping Routing Rules

Add Recordkeeping Environment

FIS

HRMS

Purchasing

OneS

tart

Recordkeeping

Inbox

WorkflowEngine

Preference Engine

EDEN(Infrastructure)Portal

(User Interface)

Applications

Conceptual Design – Workflow and Electronic Recordkeeping

RECORDS MANAGEMENT APPLICATIONS (RMA)

TRIM Attempts to include the functionality of: EDMS to provide integration with desktop

applications, add version control, and collect key documents

Records Management System to file records within the framework of the record management architecture and to manage records over their life cycle

Knowledge Management to build relationships between objects to enhance retrieval

Workflow to combine processes and objects

TRIM TRIM manages and integrates both electronic and physical records

TRIM supports the capture and import of e-mail messages and their associated attachments

TRIM Control Mechanisms: Registration – Provide evidence that a record

has been created or captured in a recordkeeping system

Classification: TRIM has a Record Plan or File Classification

module that allows organizations to build a classification system that reflects the business processes

TRIM also has a concept of Record Types to allow for further classification

TRIM Retention and Disposition: Classification scheme is linked

to Disposal Schedules TRIM allows organizations to

import disposal schedules or to create their own

Schedules are attached to records to allow for automated disposal or retention

TRIM Thesaurus: TRIM has a Thesaurus

module that conforms to the ISO standard

TRIM allows organizations to build a thesaurus or to import thesaurus terms for an accepted thesaurus

TRIM Access control is provided

through a combination of individual “Administration Access” rights, view and edit privileges assigned via the record plan, and system option setup assignments.

TRIM

Migration and Conversion

TRIM provides the tools to migrate records to another relational database system

TRIM Tracking: Audit Trails TRIM offers the capability to perform two levels

of audit logging. Core logging includes logging of the record title, number and container changes, record movements, record deletions, and all electronic activity. Full logging captures the elements of core logging plus record creations, changes to security, notes, retention schedules and triggers, dispositions, thesaurus terms, record plans, location security, location deletions, and workflow processing.

Physical Tracking of Paper Records: Barcoding, Location Designations, Movement History

TRIM TRIM provides an Application

Programming Interface (API) toolkit to allow integration with many other business system applications in an effort to provide an integrated environment

TRIM TRIM 4.3 was tested and certified as

complaint with DoD 5015.2 Standard in 1999 and again in 2001.

Statement from 1999 DoD compliance test: TRIM “integrates electronic document management and records management and provides a single interface to manage organizational records, including electronic and non-electronic records.”

http://jitc.fhu.disa.mil/recmgt/#standard

OTHER STAND ALONE RMA PRODUCTS

ForeMost Enterprise, Version 2 by TrueArc, Inc.

Tarian eRecords Engine v1.0 - formerly e-Records v1.0 by Tarian Software, Inc.

iRIMS 2001 by Open Text Corp. OBJECTIVE 2000 by Objective Corp. Hummingbird RM Family 4.0 by

Hummingbird, LTD FileSurf 7.0 by MDY Advanced

Technologies, Inc.

R2M (Relativity Records Manager) Version 2.0

by Relativity, Inc. EDMS product that adds Records Management

Functionality R2M incorporates records management seamlessly

into a document management environment. The environment is open structured and can be presented as defined by the document and records managers.

All documents, including records, must be brought into the document management environment prior to any other activity being taken upon them. As implemented, a record in R2M is simply a specially managed document. R2M extends the concept of the document lifecycle to include record dispositions.

eManage 2000 by ByteQuest Technologies, Inc.

Knowledge Management product that adds Records Management Functionality

eManage incorporates records management seamlessly into its knowledge management environment. The environment is presented as a file structure or file tree. The records manager and systems administrator work together to develop and implement a file plan as a branch or branches of the organization's knowledge file tree.

Integrated RMA Products

STAR/RIMS-E by Cuadra Associates, Inc.

Integrated product using two products from the same company

STAR is a network database management system environment in which Cuadra has developed many other information and archives management applications.

STAR/RIMS-E is a customizable application developed by Cuadra Associates, Inc. in their STAR environment. The system is very flexible and can address large, complex Records Management problems. The system supports the management of large stores of paper and non-electronic documents, as well as e-mail and other electronic documents -- all within the same environment.

Integration of FileNET IDM Content Services 5.1.1 and ForeMost

Enterprise 2.0

FileNET/ForeMost is an integrated product that combines the document management capabilities of FileNET IDM Content Services with the records management capabilities of ForeMost Enterprise. ForeMost provides the records management functionality for the pairing and uses the FileNET repository for storing records filed from FileNET.

IBM e-Records Solution (IeRS) version 1.0

by IBM Corporation IBM’s IeRS is a combination of IBM’s Content

Manager v7.1 and Tarian Software’s Tarian e-Records (TeR) v1.0.

IBM's Content Manager provides document management and workflow capabilities. It also provides the declare, search, retrieve functions, and records repository for this solution.

TeR v1.0 is a web-based RMA and provides the records management and access portions of the solution.

Work Manager/ForeMostby Eastman Software and TrueArc,

Inc. Integrated Product Work Manager/ForeMost is an integrated

product that combines the document management, imaging and workflow capabilities of the Work Manager Suite with the records management capabilities of ForeMost Enterprise. The result is a system that stores and manages both official records and their predecessor “draft” documents to provide appropriate control over the full lifecycle of the items, from origination to disposition.

OmniTREEV v2.3 by TREEV, Inc. with ForeMost Enterprise Version 1.0 by TrueArc

OmniTREEV/ForeMost is an integrated product that combines the document management, imaging, and workflow capabilities of the OmniTREEV environment with the records management capabilities of ForeMost Enterprise.

iRIMS v7.1.7 by Open Text Corporation

with DocuPACT 2000 by InterTech In the integrated product, iRIMS

provides the specialized records management functionality, while DocuPACT provides the NT File System-based electronic document/record repository, storage of document/record metadata in a relational database, and document/record search capabilities.