data integration problem how to access data across 22 different data systems, most operating on...
TRANSCRIPT
Data Integration Data Integration Problem Problem
Data Integration Data Integration Problem Problem How to access data across How to access data across 2222
different data systems, most different data systems, most operating on different hardware, operating on different hardware, using different software, and using different software, and having no common data having no common data interchange language.interchange language.
How to access data across How to access data across 2222 different data systems, most different data systems, most operating on different hardware, operating on different hardware, using different software, and using different software, and having no common data having no common data interchange language.interchange language.
HardwareHardware Operating Operating environmentenvironment
IBM 360IBM 360 360 RPG, COBOL360 RPG, COBOL
MainframeMainframe VSAM, VSE, VSAM, VSE, OS390OS390
AS400AS400 AIX, RPG, COBOLAIX, RPG, COBOL
SUNSUN UNIX, LINUX UNIX, LINUX
PCPC DOS, Windows, DOS, Windows, OS2OS2
Resource CostResource CostResource CostResource Cost
Problem: Problem: How to pull relevant information How to pull relevant information from disparate systems using from disparate systems using different software and hardware different software and hardware and report formats, into a and report formats, into a common report.common report.
Solution:Solution:Web servicesWeb services
Problem: Problem: How to pull relevant information How to pull relevant information from disparate systems using from disparate systems using different software and hardware different software and hardware and report formats, into a and report formats, into a common report.common report.
Solution:Solution:Web servicesWeb services
Resource CostResource CostResource CostResource Cost
Using old technology, linking Using old technology, linking systems required the development systems required the development of an elaborate superstructure of of an elaborate superstructure of hardware and software built around hardware and software built around specific exchanges between specific specific exchanges between specific systems using lots of client side systems using lots of client side software. The expense of these software. The expense of these systems in both personnel and systems in both personnel and financial resources was tremendous. financial resources was tremendous.
Using old technology, linking Using old technology, linking systems required the development systems required the development of an elaborate superstructure of of an elaborate superstructure of hardware and software built around hardware and software built around specific exchanges between specific specific exchanges between specific systems using lots of client side systems using lots of client side software. The expense of these software. The expense of these systems in both personnel and systems in both personnel and financial resources was tremendous. financial resources was tremendous.
Resource CostResource CostResource CostResource CostDuring the last 18 months, new During the last 18 months, new software tools have become available software tools have become available that for the first time allow ‘practical’ that for the first time allow ‘practical’ solutions to linking systems. These solutions to linking systems. These tools make it easy for users to build tools make it easy for users to build linkages to dissimilar systems using linkages to dissimilar systems using Web services and XML instead of Web services and XML instead of specific client side software solutions. specific client side software solutions. The cost of doing this has dropped by a The cost of doing this has dropped by a factor of 10. What used to cost factor of 10. What used to cost 1M$1M$ to to do can now be done for do can now be done for 100K100K. .
During the last 18 months, new During the last 18 months, new software tools have become available software tools have become available that for the first time allow ‘practical’ that for the first time allow ‘practical’ solutions to linking systems. These solutions to linking systems. These tools make it easy for users to build tools make it easy for users to build linkages to dissimilar systems using linkages to dissimilar systems using Web services and XML instead of Web services and XML instead of specific client side software solutions. specific client side software solutions. The cost of doing this has dropped by a The cost of doing this has dropped by a factor of 10. What used to cost factor of 10. What used to cost 1M$1M$ to to do can now be done for do can now be done for 100K100K. .
Resource CostResource CostResource CostResource CostWith the new Web based technologies With the new Web based technologies and standards it is possible to and standards it is possible to dramatically accelerate the pace of the dramatically accelerate the pace of the development process; drastically reduce development process; drastically reduce the cost in manpower, software, and the cost in manpower, software, and hardware; and greatly simplify the hardware; and greatly simplify the adaptability of the solution to new adaptability of the solution to new demands and needs. demands and needs.
The difference in the scale of the solution The difference in the scale of the solution that is now possible compared with what that is now possible compared with what was possible before introduction of these was possible before introduction of these Web technologies is analogous to the Web technologies is analogous to the difference between a difference between a minicomputerminicomputer from from 1980 and a 1980 and a laptoplaptop computer of 2003. computer of 2003.
With the new Web based technologies With the new Web based technologies and standards it is possible to and standards it is possible to dramatically accelerate the pace of the dramatically accelerate the pace of the development process; drastically reduce development process; drastically reduce the cost in manpower, software, and the cost in manpower, software, and hardware; and greatly simplify the hardware; and greatly simplify the adaptability of the solution to new adaptability of the solution to new demands and needs. demands and needs.
The difference in the scale of the solution The difference in the scale of the solution that is now possible compared with what that is now possible compared with what was possible before introduction of these was possible before introduction of these Web technologies is analogous to the Web technologies is analogous to the difference between a difference between a minicomputerminicomputer from from 1980 and a 1980 and a laptoplaptop computer of 2003. computer of 2003.
Portal Approach Portal Approach Portal Approach Portal Approach Linking multiple databases together in a common search environment to eliminate the need to query each system
individually.
Linking multiple databases together in a common search environment to eliminate the need to query each system
individually.
County jailsIncident reportsAdult offenderCourt ReferralDA’s system
NCIC / NLETS
Presentation to Justice Information Sharing Conference November, 2003
SSouthwestouthwest AAlabamalabama IIntegratedntegrated CCriminal-justiceriminal-justice SSystemystem
A service-oriented architecture is A service-oriented architecture is essentially a collection of services. These essentially a collection of services. These services communicate with each other. The services communicate with each other. The communication can involve either simple communication can involve either simple data passing or it could involve two or data passing or it could involve two or more services coordinating some activity. more services coordinating some activity. Some means of connecting services to Some means of connecting services to each other is needed.each other is needed.Service-oriented architectures are not a Service-oriented architectures are not a new thing. The first service-oriented new thing. The first service-oriented architecture for many people in the past architecture for many people in the past was with the use of DCOM or Object was with the use of DCOM or Object Request Brokers (ORBs) based on the Request Brokers (ORBs) based on the CORBA specification.CORBA specification.
A service-oriented architecture is A service-oriented architecture is essentially a collection of services. These essentially a collection of services. These services communicate with each other. The services communicate with each other. The communication can involve either simple communication can involve either simple data passing or it could involve two or data passing or it could involve two or more services coordinating some activity. more services coordinating some activity. Some means of connecting services to Some means of connecting services to each other is needed.each other is needed.Service-oriented architectures are not a Service-oriented architectures are not a new thing. The first service-oriented new thing. The first service-oriented architecture for many people in the past architecture for many people in the past was with the use of DCOM or Object was with the use of DCOM or Object Request Brokers (ORBs) based on the Request Brokers (ORBs) based on the CORBA specification.CORBA specification.
Services Oriented Services Oriented ArchitectureArchitecture
Services Oriented Services Oriented ArchitectureArchitecture
Presentation to Justice Information Sharing Conference November, 2003
SSouthwestouthwest AAlabamalabama IIntegratedntegrated CCriminal-justiceriminal-justice SSystemystem
ServicesServicesServicesServicesServicesServices are what you connect together are what you connect together using Web services. A service is the using Web services. A service is the endpoint of a connection. Also, a service endpoint of a connection. Also, a service has some type of underlying computer has some type of underlying computer system that supports the connection system that supports the connection offeredoffered
Web services Web services technology is the most likely technology is the most likely connection technology used in service-connection technology used in service-oriented architectures. Web services oriented architectures. Web services essentially use essentially use XML XML to create a robust to create a robust connection. connection.
ServicesServices are what you connect together are what you connect together using Web services. A service is the using Web services. A service is the endpoint of a connection. Also, a service endpoint of a connection. Also, a service has some type of underlying computer has some type of underlying computer system that supports the connection system that supports the connection offeredoffered
Web services Web services technology is the most likely technology is the most likely connection technology used in service-connection technology used in service-oriented architectures. Web services oriented architectures. Web services essentially use essentially use XML XML to create a robust to create a robust connection. connection.
MainframeMainframe
CourtsCourts
XMLXML
XMLXML
UnixUnix
Web services - The Web services - The New Data New Data
Interchange StandardInterchange Standard
Web services - The Web services - The New Data New Data
Interchange StandardInterchange Standard
Web serverWeb server
XMLXML
HostHostIntegrationIntegration
XMLXMLProbationProbation
XMLXML District AttorneyDistrict Attorney
XMLXMLLaw EnforcementLaw Enforcement
Simple ObjectSimple ObjectAccess ProtocolAccess Protocol
Web Service consumers can send and Web Service consumers can send and receive messages using SOAP receive messages using SOAP
Web ServicesWeb ServicesDescription LanguageDescription Language
Web Services are defined in terms of the Web Services are defined in terms of the formats and ordering of messagesformats and ordering of messages
Universal Description,Universal Description,Discovery and IntegrationDiscovery and Integration
You can ask a site or device for a You can ask a site or device for a description of its Web Services, or use description of its Web Services, or use a directory service such as UDDIa directory service such as UDDI
All these capabilities are built using All these capabilities are built using open Internet protocols open Internet protocols
XML, HTTP, SMTPXML, HTTP, SMTP
Open Internet Protocols
Web Web ServiceService
A programmable application A programmable application component accessible via standard component accessible via standard Internet protocolsInternet protocols
UDDIUDDI
WSDLWSDL
SOAPSOAP
What is a Web What is a Web Service? Service?
What is a Web What is a Web Service? Service?
Loosely-coupled Loosely-coupled ComputingComputing
Loosely-coupled Loosely-coupled ComputingComputing
Web servicesWeb services allow the exchange of information across public networks. There is NO requirement for special
new infrastructure as long as standards are followed.
Web servicesWeb services allow the exchange of information across public networks. There is NO requirement for special
new infrastructure as long as standards are followed.
DynamicDynamic
DynamicDynamicD
ynam
icD
ynam
ic
Dyn
amic
Dyn
amic
Dynamic
Dynamic
State county
city
Federal
Internet
LETS Portal
Desktop
Mobile
Requests from Other states
Secure requests
Justice XML
Courts
Prisons
Police
Web Services
DPS
Others
PDA
Secure requests
Pull side
Push Side
State Users
OthersDA
JMS RMS
Multiple level security
Cell phone
Problem:Problem:Court Criminal Case management SystemCourt Criminal Case management System
25 year legacy IBM mainframe25 year legacy IBM mainframe VSAM ‘flat files’ VSAM ‘flat files’ No macro data for indexingNo macro data for indexing
Solution: Solution: Daily port of VSAM files to SQL Daily port of VSAM files to SQL tables tables Stored procedure (internal DB Stored procedure (internal DB service) embedded in SQL tablesservice) embedded in SQL tables
This is a ‘hybrid’ service solution but This is a ‘hybrid’ service solution but it leaves the legacy system in placeit leaves the legacy system in place
Problem:Problem:Court Criminal Case management SystemCourt Criminal Case management System
25 year legacy IBM mainframe25 year legacy IBM mainframe VSAM ‘flat files’ VSAM ‘flat files’ No macro data for indexingNo macro data for indexing
Solution: Solution: Daily port of VSAM files to SQL Daily port of VSAM files to SQL tables tables Stored procedure (internal DB Stored procedure (internal DB service) embedded in SQL tablesservice) embedded in SQL tables
This is a ‘hybrid’ service solution but This is a ‘hybrid’ service solution but it leaves the legacy system in placeit leaves the legacy system in place
Service Solution Service Solution ExampleExample
Service Solution Service Solution ExampleExample
Court Criminal Case Court Criminal Case Management SystemManagement SystemCourt Criminal Case Court Criminal Case
Management SystemManagement System
Problem:Problem:Oracle Jail Management SystemOracle Jail Management System
Running on SUN serverRunning on SUN server
UNIX OSUNIX OS
Solution: Solution:
Middleware SUN One Application Middleware SUN One Application ServerServer
Talks to Apache web server via Talks to Apache web server via XML XML
Problem:Problem:Oracle Jail Management SystemOracle Jail Management System
Running on SUN serverRunning on SUN server
UNIX OSUNIX OS
Solution: Solution:
Middleware SUN One Application Middleware SUN One Application ServerServer
Talks to Apache web server via Talks to Apache web server via XML XML
Service Solution Service Solution ExampleExample
Service Solution Service Solution ExampleExample
Service Solution Service Solution ExampleExample
Service Solution Service Solution ExampleExample
Problem: Problem: Driver’s license photo resides on a Driver’s license photo resides on a relational DB on a separate server. relational DB on a separate server. Solution: Solution:
DB format required a script to DB format required a script to be written to pass the DL number be written to pass the DL number to the DB to retrieve the phototo the DB to retrieve the photo In this case the Script is the In this case the Script is the serviceservice
Problem: Problem: Driver’s license photo resides on a Driver’s license photo resides on a relational DB on a separate server. relational DB on a separate server. Solution: Solution:
DB format required a script to DB format required a script to be written to pass the DL number be written to pass the DL number to the DB to retrieve the phototo the DB to retrieve the photo In this case the Script is the In this case the Script is the serviceservice
.NET Devices.NET Devices.NET Devices.NET Devices
PCs and LaptopsPCs and Laptops
Pocket PCsPocket PCs
Tablet PCsTablet PCs
Cell phones and Cell phones and PagersPagers
Custom DevicesCustom Devices
Others???Others???
PCs and LaptopsPCs and Laptops
Pocket PCsPocket PCs
Tablet PCsTablet PCs
Cell phones and Cell phones and PagersPagers
Custom DevicesCustom Devices
Others???Others???
Multi-LanguageMulti-LanguageMulti-LanguageMulti-LanguagePerlPerlPythonPythonCOBOLCOBOLJscriptJscriptEiffelEiffelHaskellHaskellPascalPascalAPLAPL
PerlPerlPythonPythonCOBOLCOBOLJscriptJscriptEiffelEiffelHaskellHaskellPascalPascalAPLAPL
FortranFortranC++C++Visual Visual BasicBasicC#C#SmallTalkSmallTalkOberonOberonMercuryMercuryRPGRPG
FortranFortranC++C++Visual Visual BasicBasicC#C#SmallTalkSmallTalkOberonOberonMercuryMercuryRPGRPG
Programs created in any of these computer languages (and others) can be easily accessed by web services as a part of a larger Service Oriented Architecture Scheme.
Programs created in any of these computer languages (and others) can be easily accessed by web services as a part of a larger Service Oriented Architecture Scheme.
SOA Examples SOA Examples SOA Examples SOA Examples
User User communitiescommunities
Web Web servicesservices
PCs and PCs and Smart Smart DevicesDevices
ServersServers
ProbationProbation
NotificationNotification
Ap
plic
atio
n
Ap
plic
atio
n
Cen
ter
Cen
ter
SQ
L S
erve
rS
QL
Ser
ver
Mai
l Ser
ver
Mai
l Ser
ver
Sec
uri
ty
Sec
uri
ty
Ser
ver
Ser
ver
SQ
L S
erve
r S
QL
Ser
ver
2000
2000
DB
2D
B2
Ora
cle
Ora
cle
VS
AM
VS
AM
Mai
nfr
ame
Mai
nfr
ame
Enterprise ServersEnterprise Servers
ArrestArrest
HistoryHistory
PartnersPartners
IdentityIdentity
LETSLETS CourtsCourts PolicePolice ProbationProbation CorrectionsCorrections
SecuritySecurity