enterprise apis smarter way... · overcoming barriers to enterprise integration ... discovering and...
TRANSCRIPT
the smarter
way to
manage enterprise
APIs for
SYSPRO
ebook
eBook Chapter Menu
1. Overview……………………………………………………….………. 3
2. Overcoming Barriers to Enterprise Integration…………..…………. 4
3. Working with Mobile APIs……………………………….…………… 9
4. Accelerating Syspro Integration……………….………….……….. 10
5. Leveraging Space Based Architecture…………………………….. 13
6. Essentials of Integration Security………………….……………… 17
7. Best Practices for Choosing API Integration Services……….…. 21
8. Integration Gap Analysis Checklist……………………………….. 22
Overview
To find the smarter way to extend enterprise apps, you need to look at what the most effective IT organizations do to manage the challenges of API integration complexity, security, scalability, and resource demands. This ebook includes the Integration Gap Analysis Checklist.
Four (4) keys to
overcoming barriers to
enterprise integration
APIs are by nature designed to facilitate
movement of data and triggering of processes that are both inbound and outbound. Each API is capable of doing this for its particular domain. APIs conform to internal data definitions and application designs but not external ones. While they may take information in and out in a simplified representation or format, such as a RESTful API using HTTP protocol, the syntax of the notation is always unique and possibly even proprietary. This generates domain dissonance between APIs. APIs by themselves are not enough. APIs require intermediation to work with one another.
Event Driven
Architecture Since APIs do not have
mutual awareness, an integration platform for API management should use Event Driven Architecture (EDA) that allows business process orchestration to occur according to business rules. Event triggers, event polling and scheduling should coexist with messaging approaches such as publish and subscribe, store and forward, acknowledge on delivery, and request and reply.
Data
Transformation Since APIs do not have
common data definition languages, data mapping and transformation are required elements of an integration platform. Choose a system that supports everything from flat files, XML and databases to flow variables, operational metadata stores and user defined storage for both source and destination. Transformation expressions should include numeric, string and logical functions and operators.
Trans-Protocol
Processing Since APIs do not all use
the same transport mechanisms or protocols, use of an integration platform will overcome communication barriers between systems. An integration platform should provide communication via SOAP and REST Web Services, messaging queues (JMS, MSMQ, Websphere MQ Series), and other transport protocols (EDI, HTTP, FTP) and component architecture environments (.NET, COM, J2EE, etc.)
Process
Notation Since APIs are not self
documenting and are not
easily understood by
business users, an API
integration platform should
provide excellent process
notation and business
process modeling tools.
Models generally fall into
two categories: reference
notation and executable
notation. Finding the right
balance between fine-
grained and coarse-grained
services in the object
modeler will enable
business process designs to
come together most
efficiently.
Working With Mobile APIs
• Mobile APIs fall into three categories: • Mobile client-side APIs
• Mobile server side APIs
• Enterprise application mobile support APIs
• Mobile client-side APIs allow you to extend and enhance the capabilities of your mobile app using an API, such as a bar code scanning API.
• Mobile server-side APIs allow the application server to communicate using mobile protocols, such as an SMS communications API.
• Enterprise application mobile support APIs allow an enterprise application to interact with a mobile protocol, such as geo-location data sent from (or to) a mobile device with GPS.
Does SYSPRO have an API?
• Yes, SYSPRO ERP supports interfaces and integration through a variety of methods depending on version and requirements:
• SYSPRO e.net provides a variety of XML interfaces covering essential areas of ERP functionality and data including:
• Sales Order Query
• Sales Order Posting
• AR Posting
• AR Query
• System Setup
• System Query
• PO Query
• PO Posting
• AP Posting
• AP Query
• Inventory Posting
• Inventory Query
What are the best methods to work with the
SYSPRO APIs?
• With SYSPRO, try to find a connector that supports finding,
discovering and running COM and XML as supported by the
SYSPRO e.net API.
What does an integration platform need to
operate with SYSPRO?
• An integration platform designed to work with SYSPRO will have the following capabilities: • Ability to call and reply to COM objects.
• Ability to transform, format and transport XML.
• Ability to composite application logic including .NET, and COM.
• Ability to monitor directories, parse XML files and perform basic CRUD operations on XML files.
• Business Process Modeling.
• Business Process Orchestration (Real-time, asynchronous and scheduled).
• Automated Data Mapping and Transformation.
• Business Process Monitoring.
• Enterprise Security Compliance including optional deployment behind firewall (on-premise).
• Enterprise-Grade Scalability.
• SOA compliance and comprehensive technology protocol and adapter library.
• Mobile support for multiple platforms (iOS, Android, BlackBerry, Windows).
Leveraging three (3) Secrets of Space Based
Architecture for more scalable API messaging
dynamic scaling colocation write behind
Elastic
Partitioning
and
User Spaces
In-Memory
Data Grid
(IMDG)
Mirror
Service
Dynamic Scaling
• Using a middleware with an in-memory data grid allows APIs to be served by multiple server processes running on multiple machine instances (physical or virtual). A Space works together to store large amounts of data in memory and achieve high performance, dynamic scaling, and fail-safe redundancy. Data and business logic in the Space replicate and partition across all machines in the data grid, providing high availability even when machines or software fails.
Leveraging three (3) Secrets of Space Based Architecture for more scalable API messaging
Co-location
• With co-location of data and processing, API processes execute as they enter the system. This reduces multiple network hops and the overhead of serialization and de-serialization. Co-location reduces transport requirements, thereby aiding elasticity through straightforward scalability and maintenance. Real time clients are served by in-memory data aggregators and counters.
Leveraging three (3) Secrets of Space Based Architecture for more scalable API messaging
Write Behind
• In-Memory Computing that leverages write-behind should provide reliable asynchronous persistency. This approach allows you to asynchronously delegate the operations conducted with an In-Memory-Data-Grid (IMDG) into a backend database, significantly reducing the performance overhead. Use write behind to avoid the performance disadvantages normally associated with writing information to a database. Integration API metadata archived in a data store should be saved in background batch processes (write-behind or archiving).
Leveraging three (3) Secrets of Space Based Architecture for more scalable API messaging
APIs and the four (4)
essentials of Integration
Security
• Review the APIs to be used in your integration scenario for compliance with security practices for user authentication. Acceptable approaches may include passing software-specific application user authentication or status from the source event to the destination API, using LDAP and Active Directory, or similar approaches to authenticate and identify the in a particular business process integration flow.
Essential # 1: User Authentication
APIs and the four (4)
essentials of Integration
Security
• Review the APIs to be used in your integration scenario for compliance with security practices for user rights. Normally, the user rights are assigned and ,managed in each enterprise system. These may be assigned to individual users or to an integration server acting as a generic user. If access to the triggering application is secure, then it is generally safe to assign group rights to the integration server. Solving user access issues at the application level for each application is the best way to manage user authentication and security.
Essential # 2: User Rights
APIs and the four (4)
essentials of Integration
Security
• All middleware layers that interface
with an Application API must be
validated for their ability to employ
both encryption and decryption for
data and metadata. Generally,
middleware should support multiple
options for secure encryption such as
the RSA, DES3, BLOWFISH, RC4,
RC5, RC2, IDEA and CAST methods.
For a discussion of selecting the best
methods for your situation, see Magic xpi Security Topic: Encryption/Decryption.
Essential # 3: Ciphers
APIs and the four (4)
essentials of Integration
Security
• All middleware layers that interface with an Application API must be validated for their ability to provide an audit trail for traceability. The best practice approach to meeting this requirement will include capabilities for individual business process flow metadata, user logging and encrypted data attachments with date/time stamping. Monitors and logs may become useful for transparency and traceability in IT, financial and forensic audits.
Essential # 4: Audit
Three (3) best practices
for choosing API
integration services
• Evaluating your integration services
provider is an important part of
selecting an integration platform for
your API integration project. Be
intentional about evaluating your
services provider for support
capabilities, training offering and
overall integration experience.
Compare Service Offerings to Your
Circumstances
Service Requirements Definition
Formal definition of services needed from consulting,
project management,
implementation, testing,
documentation, training and review
perspectives.
Compare fixed-price versus time
and material approaches. Is
vendor willing to use your staff in the
project? Do they offer mentoring and take a collaborative
approach?
Determine service provider flexibility. Are they willing to
provide services as desired on-site, off-site, near-shore or off-shore? If not, why? What is the
process for change requests?
Thoroughly Evaluate Support, Training and
Experience
tech
nic
al su
pp
ort
• thorough documentation
• direct vendor telephone support
• formal online service request and response mechanism
inte
gra
tio
n tra
inin
g
• classroom courses
• online training
• formal curriculum
• mentoring
imp
lem
en
tatio
n e
xp
erie
nce
• defined but flexible methodology
• reference accounts
• experienced in wide range of APIs
Look for ERP Domain Expertise
While integration platform vendors will tend to have extensive API
integration experience, they may need to team with local service providers
experienced in your particular ERP system. The combination of local ERP
expertise and direct vendor integration support is often an ideal combination
in support of an API integration project.
Integration Gap Analysis Checklist
Implementing an interface between two
different enterprise systems requires
careful preparation. Before one can begin
an ERP integration project, preliminary
tasks need to be performed including
assembling an integration team, selecting
integration methodologies and performing a
gap/fit analysis.
Creating an integration interface between
two systems, such as SYSPRO ERP and
3rd Party ERP, CRM, HCM, SCM, PLM, or
even a Web Service such as a payroll
interface or shipment tracking Web Service,
requires an extensive understanding of the
various systems to be implemented.
Preliminary Actions
• If you are the project lead, you should make certain you have executive sponsorship so that you can get the cooperation you need from both IT and line-of-business (LOB) departments. Your sponsor will be key to making sure you’ve set aside sufficient resources. Some of those resources may need to be used to ensure you or members of your team have sufficient training. Alternatively, some work may be performed by external consultants. While this may reduce some of the requirements for extensive knowledge of the target applications to be integrated, you still need a strong enough background to make the management decisions necessary to guide the project forward successfully. In addition to training, you’ll want to assemble the vendor documentation needed by your team. Documentation will be key to identifying data formats of the systems to be interfaced.
Assembling the Team
• Project Leader. The project leader assembles the other team
members, leads the team to a decision on implementation
methodology, selects team members to fulfill the other roles on the
project, and guides the project implementation forward through the
business analysis, integration design, testing, deployment, user
acceptance and training phases.
• Technical lead. The technical lead will be responsible for
developing the interface using the selected integration methodology
for SYSPRO and the target integration environment.
• SYSPRO Functional Lead. The SYSPRO functional lead is
generally responsible for the SYSPRO implementation at your
company and contributes to the business analysis and use case
development. The lead should know about the steps involved in the
SYSPRO business processes to be automated and integrated.
• Enterprise Security Architect. A member of the team will need to
take the lead for ensuring that enterprise policies for security are
enforced. The integration methods selected should comply with
enterprise requirements for user authentication, user rights and
encryption/decryption of exchanged data during transport.
• Database administrator (DBA). The DBA(s) involved will be
responsible for table maintenance, SQL, stored procedures, and
other technical aspects of the data environment.
• Target System Functional Lead. The functional lead for the
system(s) to be integrated (regardless of whether the target system
is CRM, PLM, SCM, eCommerce, etc.) will also need to be involved
if available. The lead should know about the steps involved in the
business processes to be automated and integrated.
Establish the Scope
• Once the team has been assembled, the scope of the integration project should be defined. I recommend that the first project incorporate the minimum number of business process flows necessary for the job. Once successes are achieved by the team, additional business processes can be identified and rolled out as desired.
• When using an integration platform methodology for business process integration between SYSPRO and your target system, the gap and fit analysis will be augmented by automated means such as automated business function selections and data mappers.
• Regardless, the gap/fit analysis is a crucial step in your integration project. In this step, you will identify data mismatches and the corresponding transformations that must occur. Data mapping in the Magic xpi Integration Platform allows you to apply functions that transform data through the use of logical, numeric and textual functions.
Look at the Gap from a
Functional Viewpoint
• Take time to analyze your process from a functional standpoint. Who is responsible for the SYSPRO data and the third-party data: finance, human resources, marketing? What are the business processes involved? Identify the process, transactions and data flows. Use a business process editor and a detailed integration flow editor to chart the steps in the integration. Carefully identify and define data movements and transformations for all steps in the process. In doing so, you will find data elements in common between SYSPRO and your target system. Determine the common elements in SYSPRO and their relationships to the data in the integrated system. Specify the exact field mapping and data translation required.
Look at the Gap from a
Technical Viewpoint
• Take time to analyze your process
from a functional standpoint. Identify
the SYSPRO API functions, tables
and fields. Likewise, identify the
specific API functions, table
structure and fields in the target
system. Consult documentation as
needed and incorporate information
obtained into your project. Identify
any gaps in the data. Isolate
differences between SYSPRO and
the target system. Define any
business logic, lookups, data
transformations, exceptions flows or
other functions needed.
Examining the Integration Requirements of
Third Party Systems in Detail
File formats. Define file formats required for import such as comma separated values or physical sequences. Identify any optional fields of types. Obtain recommendations for greatest transactional integrity.
Data Transport. What data transport methods are supported by the third-party API? REST or SOAP Web Services? JMS? MSMQ? ADO.NET code? RPG calls?
Record Layouts. Identify the required data file layouts. What data, metadata and parameters is the other system expecting with each transfer of data? Do the two systems use different formats for dates? Are certain fields expected to have padded or buffered zeroes? What other special formatting requirements need to be intermediated? What other data disambiguation is required?
Event Definitions. Identify the triggers that will initiate data flows. How often do processes need to run? Will they need to be synchronous or asynchronous?
Partial vs. Full Updates. Does the target system require an entire file or just record updates?
Naming Conventions. Decide upon file naming conventions. If Directory Scans are being performed to check for file updates, what are the file names and paths?
Security and Data Privacy. Does the data need to be segregated, synchronized or archived in a particular fashion in order to be in compliance? Should data files in the integration flow be archived as part of a regulatory record keeping requirement or encrypted or erased to meet privacy regulations?
Key Data. What are the key data fields or unique identifying records that will sync the two systems together?
Mandatory Fields. Identify mandatory fields and desired optional fields. Can some data be excluded from the integration process? Does the business scenario alter what data is required (therefore requiring parametric or conditional logic)?
Examining the Integration Requirements of
Third Party Systems in Detail
File Length Limitations. Does either system impose file length limitations on the number of fields
or records?
User Defined Fields. Has either system implemented user defined fields? (Almost certainly yes!)
Are these well documented? If not, undertake a process of discovery to determine the UDF
structure of your system databases. How will user defined fields be handled by the other system?
Do user defined fields need to be created in order to accept other data from the third-party system?
Validation and Exceptions. Identify any data validation and exceptions processes performed by
either system upon import/export of data. Is an exceptions file created? How will these exceptions
be handled? Do the systems generate error conditions or messages? How will these be trapped
and handled? If data validation is not handled by the API, what validation checks do you need to
add to your integration business logic to avoid introducing errors or bad data?
Logging. What steps in the business process need to be logged and measured for latency, volume
and completion accuracy (errors)?
Process Responsibility. For processes that are triggered by user interaction, posting or closing
activities, how will the users be trained on any necessary changes to their work routines and
schedules, Do alarms, alerts and escalations need to be built into the integration flow. Does human
workflow need to be automated? Will email notifications, in-application tasks or third party human
workflow tools be invoked? Who is responsible and who manages those responsible? What is the
chain of escalation and how much automation is required or expected?
Long Running Processes. Are there any long-running processes required? What business rules
will apply for neglected workflow or abandoned business processes?
Setting Up the Sandbox. You will need to set up a test environment or sandbox with non-
production test data. Are there tools available for stress testing in the sandbox environment?
Integration between SYSPRO and the third-
party system will involve detailed data mapping.
Implement your data mapping using the
information gathered during the functional and
technical gap/fit analyses. Data mapping always
involves at least one source and at least one
destination.
Analyze your ERP transactions; carefully
identify the relationships between these
transactions and the requirements of the third-
party system.
Perform the Data Mapping
34
www.magicsoftware.com
+1 (949) 250-1718 ext. 259
Read the Integrate My JDE blog on it.toolbox.com
For Additional Information