JBoss Enterprise Data Services (Data Virtualization)
Post on 17-Aug-2015
<ol><li> 1. 1 JBoss Enterprise Data Services Peter Larsen JBoss Solutions Architect, Red Hat Inc. email@example.com </li><li> 2. 2 Agenda Motivation for Data Services EDS in the community / History Positioning Data Services in an Enterprise Architecture Use Case Domains / Customer Examples Technical Architecture Demo </li><li> 3. 3 The Business Inflexibility Trap Inflexibility: The essential Business Problem With agility all problems are solvable With enough eyes, all bugs are shallow </li><li> 4. 4 Agility is Key Because change is the only constant Technologies Requirements Regulations Standards Models Processes ??? What is the single most important problem preventing Agility? </li><li> 5. 5 Problem: Data Challenges Challenges Different physical structure Different terminology and meaning Different interfaces May need to federate/integrate May be locked in to database Must ensure performance Maintain/Improve security Tremendous value in existing information assets, but... Time consuming and costly to implement new applications that leverage this information Data Warehouse Packaged Applications Operational Data Stores Data Gap </li><li> 6. 6 Problem: Data Challenges Alternatives Time consuming difficult/costly No re-use of data logic Any changes break the application Data Gap Data Gap Hard Code Replicate/Data Mart Data not fresh Costly additional licenses More copies of data = more silos Governance/security </li><li> 7. 7 Solution: JBoss Enterprise Data Services JBoss Enterprise Data Services Data Service Data ServiceData Service SQL Web Services Access to multiple data stores in real time Standards-based read/write access Speeds application development Transform data structure and semantics Consolidates data into a single view Centralized access control Enterprise-proven flexible, scalable, high-performance Turns the Data You Have Into the Information You Need </li><li> 8. 8 Data Services Platform Where it fits JBoss Enterprise Data Services Platform Other Vendor Portal / ESB/SOA Platforms Data Service Data Service Data Service Data Service Data Service </li><li> 9. 9 Integration Technologies ETL SOA EDS Data ProcessIntegration Style DataIntegrationTimeliness Real Time Batch </li><li> 10. 11 Data Services Platform Common Use Cases Service Oriented Architecture Federate/transform data efficiently use by higher-level services Insulate business processes from data access details Business Intelligence, Operational Analytics, Reporting Consolidated financial reports/dashboards/KPIs Virtual data marts Information Consolidation, Reference Data Management Single/360 view of Customer Single/360 view of Supplier Single/360 view of Employee Regulatory Compliance Provide common security, central access and auditing of data VISA PCI, Sarbanes Oxley, Basel II, HIPAA </li><li> 11. 12 JDBC/ODBC Query Engine Data Virtualization, Federation JBoss Enterprise Data Services JBoss ModeShape Repository Services JBoss jBPM JBoss Rules JBoss Enterprise SOA Platform JBossESB JBoss Enterprise Application Platform Red Hat Enterprise Linux Windows, UNIX, other Linux Turns the data you have into the information you want Augments and extends SOA Platform to address data access, integration and abstraction. SOA Patterns, best practices Reporting/Analytics enablement Information Consolidation, Data Mgmnt Data Governance, Compliance Real-time read/write access to heterogeneous data stores Speeds application development by simplifying access to distributed data Centralized access control, auditing JBoss Enterprise Data Services </li><li> 12. 13 </li><li> 13. 14 Data Services Platform Architecture </li><li> 14. 15 Query Performance & Optimization Minimal overhead for simpler requests Control enforce mandatory criteria with certain requests enforce time and size limitations on requests Rule-based optimization use criteria to avoid unnecessary fields and records removal of unnecessary joins across data sources merge all transformation logic for a single source Cost-based optimization join algorithms (nested loop, merge, dependent, hash) cost profile of each data source Data caching and staging (materialized views) Manage dataflow buffer management </li><li> 15. 16 Designer Tooling Virtual Models Physical Models representing actual data sources Shows structural transformations Defines transformations with Selects Joins Criteria Functions Unions User Defined </li><li> 16. 18 Semantic Mediation/Integration T Authoritative Sources: Mapped to logical view Multiple Internal/External Information Sources Application views of information: Relational, XML, Java T T XML Document T T T Web Services Web Services Workflow/ ESB Workflow/ ESB Business Applications Business Applications Claims, Billing, Policies, bldg_id SITENUM Facility_ID Location_ID bldg_type Depot_Number Location_Type Semantic Data Services Data Dictionary: Based on logical data model or XML schema Support for multiple COIs Support for multiple versions </li><li> 17. 19 Data Services and the ESB User-facing Logic (Service Consumers) Business Logic Data Logic Process and Other Integration Logic Rich or Thin Desktop Process, Integraion Services Business Services ESB Direct ODBC, JDBC ODBC, JDBC WSDL, SOAP, MOM, other WSDL, SOAP, MOM, other Process Orchestration Services Data Services </li><li> 18. 20 JBoss Enterprise SOA Platform Enables Business Process Automation by integrating and orchestrating application components and services running on JBoss Enterprise Middleware and/or any other standards-based AS Single distribution that integrates JBoss ESB, jBPM, JBoss Rules, Enterprise, Application Platform Enables multiple integration styles: SOA integration, EAI, EDA, process and business rules technologies to automate business processes to improve business productivity Certified Platform for Service Integration and Orchestration Simple, Flexible, and Scalable Light footprint, simple installation JBoss ON platform management and services monitoring Scalable clustering to support high transaction volumes A flexible, standards-based platform to integrate applications, SOA services, business events and automate business processes. Red Hat Enterprise Linux Windows, UNIX, other Linux Workflow Rules JBoss Enterprise SOA Platform JBossESB Transformation, Routing, Registry JBoss Enterprise Application Platform Container services, Hibernate, Web Services stack, Seam, Clustering, Cache, Messaging, Transactions </li><li> 19. 22 Enterprise Data Services 5.2 EDS 5.2 Released December 2011 Tighter data services/ESB tooling integration Performance tweaks LOB handling Cost-based optimizer enhancements Programmatic view creation Repository enhancements Versioning support More artifact types Cloud-based data sources Fixes, minor enhancements, additional platform certs </li><li> 20. 23 Business Value of Enterprise Data Services Greater agility, faster time to solution Increased ROA Improved organizational performance Better control of information Improved utilization of data assets Derive more value from existing investments Complements existing systems Jumpstart Your SOA Initiatives! Better/faster than hand coding Faster, less costly than data replication Data virtualization provides loose coupling The right data at the right time to the right people Decision support, BI with a complete view of information across the enterprise Powerful security, Auditing, Data Firewall Avoid data silo proliferation Central data access and policy, Compliance </li><li> 21. 24 A Comprehensive Middleware Portfolio JBoss Enterprise Data Services Platform JBoss Enterprise SOA Platform JBoss Enterprise Application Platform JBoss Enterprise Web Platform JBoss Enterprise Web Server Red Hat Enterprise Messaging JBoss Enterprise Portal Platform JBoss Enterprise Business Rules Management System JBossDeveloper Studio Seam Hibernate WebFramework Kit JBoss Operations Network RedHatServices Cloud Implementation Cloud GovernanceCloud Strategy & Selection VMWare Microsoft Hyper-V Red Hat Enterprise Virtualization PrivatePublic Amazon EC2 Other RHEL,Unix,Windows </li><li> 22. 25 Where did Teiid come from? Project lineage is from MetaMatrix starting in ~1999. Teiid - http://www.jboss.org/teiid Teiid Designer - http://www.jboss.org/teiiddesigner DNA - http://www.jboss.org/dna/ MetaMatrix was the leader in Enterprise Information Integration (EII) hence Teiid. Red Hat acquired MetaMatrix in 2007. Last major MetaMatrix product release, 5.5.4 11/09 </li><li> 23. 26 Project Status (March 2011) Open source 2/2009 heavily refactored from 5.5 line 7.0 Initial release 6/2010 7.1 Teiid / Teiid Designer release 8/2010 Basis for EDS 5.1 release with hundreds of issues resolved and targeted enhancements 7.4 Coming Soon! More source integration (MDX via XMLA, Ingres), expanded function support, etc. should be picked up along the work in 7.1-3 by the next service pack release. </li><li> 24. 27 Community Version Community web site: www.jboss.org/teiid Teiid sub-projects: Teiid Runtime, Teiid Designer Teiid 7 is built for AS7 </li><li> 25. 28 JBoss Enterprise SOA Platform (SOA-P) and Enterprise Data Services (EDS) Roadmap Q4 08 Q1 09 Q2 09 Q3 09 2.8 SOA-P 5.0 Q4 09 3.0 Q1 10 Q4'09 Q1'10 Q2'10 Q3'10 Q4'10 Q1'11 Q2'11 Enterprise Q3'11 Q4'11 Q1'12 Platform Release March 2010 Calendar Quarters Platform Release January 2011 (target) SOA-P 5.1 Platform Release January 2011 (target) SOA-P/EDS 5.2 EDS 5.1 Platform Releases Mid-Q2 2011 (target) MetaMatrix 5.5.4 Platform Release November 2009 </li><li> 26. 29 Generating New Value with JBoss Enterprise SOA Platform and Enterprise Data Services </li><li> 27. 30 The context Organizations Significant assets already deployed or otherwise in use Applications, databases, services, spreadsheets, file extracts, manual processes, tribal knowledge Not realizing full benefit Mandate Remove business impediments, improve status quo Control/reduce costs Derive greater value from the assets you already have </li><li> 28. 31 Common Challenges Data Decision making Inflexible systems Manual processes </li><li> 29. 32 Common Challenges Data Data sprawl Tied up in silos Not reconciled/integrated Not easily usable Decision making Inflexible systems Manual processes </li><li> 30. 33 Common Challenges Data Decision making Insufficiently informed Missing key information Stale or out-of-context information Inflexible systems Manual processes </li><li> 31. 34 Common Challenges Data Decision making Inflexible systems Logic hard-coded into applications Redundant logic, not standardized or shared Changes require development cycle, resources, time Unable to react quickly to business, market, IT changes Manual processes </li><li> 32. 35 Common Challenges Data Decision making Inflexible systems Manual processes Business processes are manual Data entry, swivel-chair integration Overly dependent on individuals Inconsistent, prone to error, difficult to govern </li><li> 33. 36 Common Challenges Data Decision making Inflexible systems Manual processes But... These data, systems, applications, decision-making processes, business processes and logic are your current assets waiting to be improved and put to better, more effective use. How? </li><li> 34. 37 Solution Patterns 1. Pattern: Data Foundation 2. Pattern: Information Delivery 3. Pattern: Externalize Knowledge 4. Pattern: Automate Decision Making 5. Pattern: Codify Business Processes </li><li> 35. 38 Solution Patterns: Data Foundation Liberate, integrate, mediate, transform data Tap silos, gain control over data sprawl Create foundation data layer through data virtualization xml databases warehouses spreadsheets services sale > files applications ExistingExisting sources andsources and silos of datasilos of data Integrated setIntegrated set of canonicalof canonical data objectsdata objects CRM, Employee SupplyChain, Logistics </li><li> 36. 39 Solution Patterns: Information Delivery Provide consistent information in the form required by different information consuming applications, processes, services. Ensure complete information through all delivery modes/formats. Forms: Relational Tables/Views Star schema Procedures Schema-compliant XML Access Modes: JDBC, ODBC SOAP Web Services POJO XML over HTTP, JMS (contract) (contract) (contract) Custom Apps Business Processes Packaged Apps Reports, Dashboards Data warehouses O/RMappingJDBC/OSOAP/JMS CRM, Employee SupplyChain, Logistics </li><li> 37. 40 Solution Patterns: Externalize Knowledge Externalize key business logic from application code Isolate and standardize rules that govern business decisions and operations Enable business analysts and development to collaborate in defining functional behavior Rule sets possibilities: Pricing Fraud detection Regulatory compliance Productivity/Efficiency Control systems Product configuration ... Insurance Rules: Age Sex Health Occupation = $ Price </li><li> 38. 41 Solution Patterns: Automate Decision Making Move beyond reports to active analysis and decision making Extend rule sets to analyze information provided through earlier patterns. Process information on scheduled basis or dynamically as data is flowing through applications and on the bus. Raise alerts, initiate corrective actions, seize opportunities sale > </li><li> 39. 42 Solution Patterns: Codify Business Processes Codify the processes actually followed by your organization Create standardized, reusable workflows/orchestrations Eliminate unnecessary manual steps, keep human tasks only where appropriate. Identify common business patterns both standard normal processes and exception remediation processes Extend automated decision making with business processes and vice versa </li><li> 40. 43 Solution Patterns 1. Pattern: Data Foundation 2. Pattern: Information Delivery 3. Pattern: Externalize Knowledge 4. Pattern: Automate Decision Making 5. Pattern: Codify Business Processes </li><li> 41. 44 How technologies map to patterns JDBC/ODBC Data Virtualization Data Access, Federation JBoss Enterprise Data Services Metadata Repository Repository Services Workflow Rules JBossESB Transformation, Routing, Registry JBoss Enterprise Application Platform Container services, Hibernate, Web Services stack, Seam, Clustering, Cache, Messaging, Transactions Red Hat Enterprise Linux Windows, UNIX, other Linux JBoss Enterprise SOA Platform 1. Data Foundation 2. Information Delivery 3. Externalize Knowledge 4. Automate Decision Making 5. Codify Business Processes </li><li> 42. 45 How technologies map to patterns JDBC/ODBC Data Virtualization Data Access, Federation JBoss Enterprise Data Services Metadata Repository Repository Services Workflow Rules JBossESB Transformation, Routing, Registry JBoss Enterprise Application Platform Container services, Hibernate, Web Services stack, Seam, Clustering, Cache, Messaging, Transactions Red Hat Enterprise Linux Windows, UNIX, other Linux JBoss Enterprise SOA Platform 1. Data Foundation 2. Information Delivery 3. Externalize Knowledge 4. Automate...</li></ol>
View more >
Denodo DataFest 2016: Modern Data Architectures: Deploying Data Virtualization at an Enterprise Scale
RED HAT JBOSS DATA Hadoop, Spark Databases and data warehouse Mainframe Enterprise applications Excel, CSV, ... DATASHEET Red Hat JBoss Data Virtualization 2
WS-Trust Virtualization Software + Services Data-aware Enterprise Software Factories Oslo Enterprise Services Bus Claims-Based Identity.
Red Hat JBoss Enterprise Application Platform 7.0 ... Hat Customer Content Services Red Hat JBoss Enterprise…
GAIN BETTER INSIGHT FROM BIG DATA USING JBOSS Apps JBoss Data Virtualization Hive Inventory Databases Purchase Mgmt ... Interactive SQL Queries at Petabyte Scale ... –Oracle JDK 1.6, ...
Data Virtualization – Flexible Technology for the Agile ... Virtualization – Flexible Technology for the Agile Enterprise ... Data Virtualization – Flexible Technology for the Agile Enterprise ... service$ Data$
RED HAT JBOSS DATA VIRTUALIZATION ?· facebook.com/redhatinc @redhatnews linkedin.com/company/red-hat…
ARRIS Global Services - CAS Virtualization - Data OVERVIEW : WHY VIRTUALIZE? Virtualization enables these essential features for assuring ... ARRIS Global Services - CAS Virtualization - Data Sheet
Red Hat JBoss Data Grid 6 Hat JBoss Data Grid 6.1 Developer Guide For use with Red Hat JBoss Data Grid…