Information Management Trends and Some History

Download Information Management Trends and Some History

Post on 30-Oct-2014




0 download

Embed Size (px)




<ul><li> 1. Information Management Trends and Some History C. Mohan , PhD IBM Fellow &amp; IBM India Chief Scientist Member, IBM Software Group, Asset Architecture &amp;Information Management Architecture Boards [email_address]</li></ul><p> 2. Key Customer Pain Points </p> <ul><li>Cant Find Information Discovery </li></ul><ul><li>Cant combine Information Integration </li></ul><ul><li>Cant extract value from Information Insight </li></ul><ul><li>Cant consume Information Dissemination </li></ul><p> 3. The Information Challenge Information is in Silos Trusted Information is Not Available Todays business challenges mandate a fresh approach to managing information Managing information in silos hasbecome obsolete Multiple Versions of the Truth Inaccurate, Untimely Inconsistent Incomplete, Inaccessible Out of Context Globalization, M&amp;As Risk &amp; Compliance, Eroding Customer Loyalty, Supply Chain Complexity, Industry Transformations, Cost Cutting 70%of peoples time can be spent searching for relevant information 60%+ of CEOs: Need to do a better job leveraging information Sources: IBM Attributes &amp; Capabilities Study, 2005; Client Interviews 2004; IBM CFO Study, 2006 5X More Valuecreation by organizations effective at using Information as an Asset Information Must Become a Strategic Asset 4. Information Management Trends </p> <ul><li>Information Intensive Applications </li></ul><ul><li><ul><li>Shift from transaction-centric to information-intensive applications </li></ul></li></ul><ul><li>Information Diversity </li></ul><ul><li><ul><li>Delivering insight over increasingly diverse sources of information </li></ul></li></ul><ul><li>New Business &amp; Delivery Models </li></ul><ul><li><ul><li>Information as a Service, Outsourcing, New Licensing Models</li></ul></li></ul><ul><li>Democratization of Information </li></ul><ul><li><ul><li>Changing User Expectations &amp; the Parent Test</li></ul></li></ul><ul><li>Massive Collaboration &amp; Societal Intelligence </li></ul><ul><li><ul><li>Collaboration over shared information to creating business insight </li></ul></li></ul><p> 5. Information as a Strategic Asset Presentation Services EDW Legacy Legacy Portals, Browsers, and or Devices Strategic APPL Event Processing Tactical APPL Tx APPL App Server Discovery APPL Master Data APPL ProcessServices Information Integration Services Analytic Services Master Data Services TransactionApplication Services Analytic Application Services Business Process Management Federation DiscoveryServices ECW Content Services CollaborationServices Notes Email Enterprise Service Bus MetadataServices Master data Hubs Product Customer Supplier Location TransactionServices OLTP2 OLTP1 OLTP Business Rules Business Monitoring Streaming Batch Metadata 6. Master Data Compliance&amp; RiskMgmt. Sales andMarketing Closed LoopCampaignMgmt. Customer Service Data Stewardship &amp; Administration Compliance Marketing Account Administration PrivacyManagement Web Self-Service Wireless Self-Service Distributor IVR Self-Service Branch /Sales Office Call Center Browser-based Unlimited Attributes Multiple Categorizations Multi-enterprise Standards-based Security and Audit New Business Processing Privacy and Data Mgmt. Marketing Insight Customer Facing Channels Internal Users Customer Master Data Integration 7. IBM Information Management Software Delivering Value Beyond Traditional Repositories </p> <ul><li>Data Services </li></ul><ul><li><ul><li>Databases, Warehouses, Tools </li></ul></li></ul><ul><li>Content &amp; Discovery Services </li></ul><ul><li><ul><li>Content Mgmt. &amp; Integration Services </li></ul></li></ul><ul><li><ul><li>Discovery Services </li></ul></li></ul><ul><li>Information Integration Services </li></ul><ul><li><ul><li>Quality Services </li></ul></li></ul><ul><li><ul><li>Transformation Services </li></ul></li></ul><ul><li><ul><li>Federation Services </li></ul></li></ul><ul><li><ul><li>Metadata Services </li></ul></li></ul><ul><li>Information Accelerators </li></ul><ul><li><ul><li>Master Data Management </li></ul></li></ul><ul><li><ul><li>Entity Analytics </li></ul></li></ul><ul><li><ul><li>Information Warehousing </li></ul></li></ul><ul><li><ul><li>Customizable Dashboards </li></ul></li></ul><ul><li><ul><li>Industry Data Models </li></ul></li></ul><p>Information Delivered On Demand Based on Services Oriented Architecture 8. DB2 9 A Pure XML, Relational Hybrid XML Developer I see a sophisticated XML repository that also supports SQL." SQL Developer "I see a sophisticated RDBMS that also supports XML." FamiliarProgramming Models Optimized Storage Models Mature Services FamiliarTooling Optimized Performance &amp;Scale 9. Integration of XML &amp; Relational Capabilities DB2 SERVER CLIENT SQL/XML XQuery DB2 Engine XML Interface Relational Interface XML DB2 Storage : DB2 Client / Customer ClientApplication </p> <ul><li><ul><li>Applications combine XML &amp; relational data </li></ul></li></ul><ul><li><ul><li>Native XML data type (server &amp; client side)</li></ul></li></ul><ul><li><ul><li>XML Capabilities in all DB2 components </li></ul></li></ul><p>Relational 10. DB2 V9 pureXML support XQuery SQL/XML APIs/Client XML Indexes XML Schema support Native Storage XML Load Import/Export </p> <ul><li>Native XML support in DB2 with more to come </li></ul><ul><li>Seamless integration with the relational world </li></ul><p>New XMLJoin Methods Tools And all the relational stuff 11. DB2 V9 pureXML support </p> <ul><li>XML as a native data type </li></ul><ul><li>Pure XML storage and indexing </li></ul><ul><li>XQuery and SQL/XML support </li></ul><ul><li>XML Schema Repository </li></ul><ul><li>Schema validation </li></ul><ul><li>Application Support (Java, C/C++, .NET, PHP, etc.) </li></ul><ul><li>Visual Tooling, Control Center Enhancements </li></ul><ul><li>Annotated schema shredding </li></ul><ul><li>DB2 Utilities: Import/Export, HADR, etc. </li></ul><ul><li> and more </li></ul><p>Secure and Resilient Infrastructure for a New Breed of Agile Applications DB2 9 12. Someof Our Info Mgmt Research Legacy </p> <ul><li>Invention of Relational Model/Technology &amp; SQL </li></ul><ul><li>Research prototypes </li></ul><ul><li><ul><li>System R</li></ul></li></ul><ul><li><ul><li>R* Distributed DBMS </li></ul></li></ul><ul><li><ul><li>Starburst Extensible Object-Relational DBMS</li></ul></li></ul><ul><li><ul><li>Garlic Heterogeneous DBMS</li></ul></li></ul><ul><li>Product Contributions </li></ul><ul><li><ul><li>Data sharing on DB2 390 Sysplex</li></ul></li></ul><ul><li><ul><li>DB2 UDB Query Processor</li></ul></li></ul><ul><li><ul><li>Intelligent Miner </li></ul></li></ul><ul><li><ul><li>Lotus Notes R5 Recovery </li></ul></li></ul><ul><li><ul><li>Discovery Link &amp; DB2 Information Integrator </li></ul></li></ul><ul><li>6 IBM Fellows from team of &lt; 50 </li></ul><p> 13. Why We Have Experience with Customers </p> <ul><li>Over 2 decades of partnershipwith SWG Toronto &amp; SVL </li></ul><ul><li><ul><li>Incorporation of Starburst prototype into DB2 </li></ul></li></ul><ul><li><ul><li>Component Owners of DB2 for LUWs Query Compiler </li></ul></li></ul><ul><li><ul><li>Versions 2 5 (1992-1997) </li></ul></li></ul><ul><li><ul><li>Dealt with customer APARs, Visits, &amp; Presentations </li></ul></li></ul><ul><li>Responsible for many DB2 innovations </li></ul><ul><li><ul><li>Query Graph Model (internal query representation, key to extensibility) </li></ul></li></ul><ul><li><ul><li>Query ReWrite and Optimizer technology </li></ul></li></ul><ul><li><ul><li>ARIES recovery and locking methods </li></ul></li></ul><ul><li><ul><li>Triggers and Constraints </li></ul></li></ul><ul><li><ul><li>Star Join and Hash Join </li></ul></li></ul><ul><li><ul><li>Object-relational features</li></ul></li></ul><ul><li><ul><li>Automatic Summary Tables (materialized views) </li></ul></li></ul><ul><li><ul><li>Visual Explain </li></ul></li></ul><ul><li><ul><li>Index Advisor</li></ul></li></ul><ul><li>Respected for our vision </li></ul><ul><li><ul><li>World-class publications in leading database conferences </li></ul></li></ul><ul><li><ul><li>Cognizant of industry trends </li></ul></li></ul><p> 14. Leveraging Technology and People IMS Development DB2 Development IDS / U2 Development CustomerRequirements IBM Products IBM Research 15. SVLDB2 UDB for z/OS &amp; OS/390 IMS Business Intelligence Content Management DB2 Everyplace Red Brick Icing Traditional AD Languages BoeblingenDB2 Text Extenders SAP/R3 Enablement Intelligent Miner for Data Intelligent Miner for Text Somers Hawthorne Advanced Technology Almaden Advanced Technology Austin GBIS PortlandXPS &amp; DB2 Lenexa IDS Boulder &amp; Denver Content Management U2 Datablades Boca Raton &amp; Miami EMMS LA Informix Support RochesterDB2 UDB for AS/400 TorontoDB2 UDB for UNIX,Windows, &amp; OS/2 IBM Information Management Teams Beijing Information Integration DB2 for zOS Content Management DB2 and IMS tools Las Vegas Entity Analytics Over 6000 employees worldwide Menlo Park &amp; Oakland IDS XPS JDBC Visionary Cloudscape Datablades Object Connect &amp; Translator Content Management India DB2 UDB Service Business Intelligence IDS YamatoHigh Speed Inverted Index Search Business Intelligence Content Management HursleyEnterprise Master Data Solutions</p> <ul><li>India Software Lab </li></ul><ul><li><ul><li>3000 employees </li></ul></li></ul><ul><li><ul><li>Broad range of skills all SWG Brands </li></ul></li></ul><ul><li><ul><li>Linux Competency Center </li></ul></li></ul><ul><li>DB2 Lab within ISL </li></ul><ul><li><ul><li>100+ developers</li></ul></li></ul><ul><li><ul><li>Lab based services teams DB2, CM, BI </li></ul></li></ul><ul><li>Other Resources </li></ul><ul><li><ul><li>India Research Lab </li></ul></li></ul><ul><li><ul><li>Solution Porting Center </li></ul></li></ul><ul><li><ul><li>Education Center for IBM Software </li></ul></li></ul><ul><li><ul><li>IBM Academic Initiative </li></ul></li></ul><p> 16. A Spectrum of Info Serving Requirements Platform: Mobile Desktop Small Servers Large ServersData Size: MicroCompact LargeExtremely Large Workload: BatchOnline TransactionsReal-time AnalysisData Mining Structure: HierarchicalRelationalMulti-ValueXML OS: SymbianPalmOSWindowsLinuxUnixi5/OSz/OS Scope: EmbeddedIntra-applicationSingle applicationMulti-application Support: None Web/E-mail Business hours 24x7 17. Products to Match the Spectrum of Data Serving Needs DB2 Everyplace OLTP Relational Mobile EmbeddedLinux PalmOS Symbian Cloudscape OLTP Relational Intra-App / Single-App Java IDS OLTP Relational Intra-App / Single-App AIX, etc. Linux Windows DB2 OLTP &amp; Analysis Relational&amp; XML Single / Multi-App z/OS I5/OS AIX, etc. Linux Windows IMS OLTP Hierarchical Single / Multi-App z/OS U2 OLTP Multi-Value Intra-App / Single-App AIX, etc. Linux Windows Superior capabilities across the spectrum of requirements 18. DB2 for z/OS </p> <ul><li>The power and function of an open, industry standard data serverwith zSeries industry leading availability, performance, and security </li></ul><ul><li>What it takes to be the industrys most extreme data server </li></ul><ul><li>Continuous application availability measured in years </li></ul><ul><li>Ability to process over1B SQLtransactions per hour </li></ul><ul><li>Uninterrupted growth from 1 byte to over a peta-byte </li></ul><ul><li>Serving 100s of applications for 100,000s of users</li></ul><ul><li>US Governments highest security classification (zSeries) </li></ul><ul><li>Support for industry standards:XML, Web services, Java, C, COBOL </li></ul><ul><li>Support for complex business applications:SAP, PeopleSoft, Siebel </li></ul><p>Extreme qualities of serviceXML and Relational data server 19. Technology Evolution with Mainframe Specialty Engines Integrated Facility for Linux (IFL) 2001 IBM System z9 Integrated Information Processor (IBM zIIP) planned for 2006 System z9 Application Assist Processor (zAAP) 2004 </p> <ul><li>Building on a strong track record of technology innovation with specialty engines, IBM intends tointroduce the System z9 Integrated Information Processor </li></ul><ul><li>Supportfornewworkloadsand openstandards</li></ul><ul><li>Designed to help improve resource optimization for eligible data workloads within the enterprise </li></ul><ul><li>Centralizeddata sharingacross mainframes </li></ul><ul><li>Incorporation of JAVAinto existing mainframe solutions</li></ul><p>Internal CouplingFacility (ICF) 1997 20. Data Challenges </p> <ul><li>Variety, Velocity, and Volume </li></ul><ul><li>New composite applications need data from multiple sources </li></ul><ul><li><ul><li>Consumers expect holistic, personalized, and value-added content </li></ul></li></ul><ul><li><ul><li>Relational, XML, packagedapplications, content repositories,file systems all contain critical business information </li></ul></li></ul><ul><li>Increasing emphasis on current data </li></ul><ul><li><ul><li>Real-time analytics </li></ul></li></ul><ul><li><ul><li>Business activity monitoring</li></ul></li></ul><ul><li>Petabytes will be the measure of available online data</li></ul><ul><li><ul><li>All client interactions are important ( e.g., instant messages, audio records, web traffic,) </li></ul></li></ul><ul><li><ul><li>Internet and intranet content</li></ul></li></ul><p>The world produces 250MB of information every year for every man, woman and child on earth. 10-100GB 100sGB - 1TB 1 - 20 GBs 100sMB 100s KB 1999 1s TB 1s TB 100s TB 100s TB 1s TB 1s TB 10s GB 10s GB 1s GB 1s GB 2004 10X 100X 100X 1,000X 10,000X Common Database Sizes Common Database Sizes Transactions Warehouses Marts Mobile Pervasive 37%CGR Disk Growth 96-07 70,000 TBof TV and Radio content in 2002 alone; 30% growth/year 21. Addressing the Changing Characteristics of Data Actionability Heterogeneity Scale Satellite &amp; Surveillance Images and Video Gene Sequences Transactions Text and Web Increasing need to manage and analyze new data types Protein Folding 22. Research in Information and Interaction Drive our leadership technologies for search, structured and unstructured information processing and analytics, natural language processing, and conversational and multimodal interaction, across multiple tiers of business activities in SWG products and solutions.Foster the exploitation of components with these leading research technologies in IGS services offerings. CM Information Retrieval NLP Analytics Video Analysis Conversational andMultimodal Interactions Unstructured Information Management Information Management Database Synthesis Information Integration Metadata Speech Recognition 23. Worlds of Structured &amp; Unstructured Data Come Together Analytical Complexity Collect Store Retrieve Drill Mine ETL Warehouse SQL OLAP Cluster, Classify, .. Crawl ECM Search Navigate Cluster, Classify, .. Solutions II Structured Data Unstructured Data 24. Need for Business Intelligence </p> <ul><li>Loyalty </li></ul><ul><li>Profitability </li></ul><ul><li>Buyer Behavior </li></ul><ul><li>Targeted Offers </li></ul><p>Homeland Security </p> <ul><li>Internet Buzz </li></ul><ul><li>Anti-Money Laundering </li></ul><ul><li>Border Control </li></ul><ul><li>Crime Information </li></ul><ul><li>Globalization </li></ul><ul><li>Business Controls </li></ul><ul><li>Mergers and Acquisitions </li></ul><ul><li>Supply Chain Efficiencies </li></ul><p>Accountability and Compliance Customer Knowledge Business Performance </p> <ul><li>Risk Management </li></ul><ul><li>Fraud and Abuse </li></ul><ul><li>Public Protection </li></ul><p>HIPAA Basel II Patriot Act Sarbanes-Oxley Capitalism and Its Troubles: A Survey of International Finance-May 24, 2002Preparing for terror How scared should you be? Nov 28th 2002From The Economist print edition 25. SOA Reference Architecture Business Innovation &amp; Optimization Services Development Services Integrated environment for design and creation of solution assetsManage and secure services, applications &amp;resources Facilitates better decision-making with real-time business information IT Service Management Infrastructure Services Optimizes throughput, availability and performance Partner Services Business App Services Access Services Connect with trading partners Build on a robust, scaleable, and secure services environment Facilitates interactions with existing information and application assets Interaction Services Process Services Information Services Enables collaboration between people, processes &amp; informationOrchestrate and automate business processes Manages diverse data and content in a unified manner ESB Facilitates communicationbetween services Apps &amp;Info Assets 26. Information as a ServiceThe SOA Lifecycle Mapped to Information Needs </p> <ul><li>Understand Information Assets and Link to Business Context </li></ul><ul><li><ul><li>Discover informationmetadata </li></ul></li></ul><ul><li><ul><li>Map information tobusiness processes </li></ul></li></ul><ul><li><ul><li>Develop data &amp;content models </li></ul></li></ul><ul><li>Compose Information Services Across Heterogeneous Sources </li></ul><ul><li><ul><li>Extract, federate &amp; transform heterogeneous information </li></ul></li></ul><ul><li>Service Information Requests </li></ul><ul><li><ul><li>Deliver unified data&amp; content </li></ul></li></ul><ul><li><ul><li>Deliver businesscontext </li></ul></li></ul><ul><li><ul><li>Discoverrelationships </li></ul></li></ul><ul><li>Ensure Performance, Availability &amp; Security Meet Service Levels </li></ul><ul><li>Define &amp; Refine Information Management Rules &amp; Policies </li></ul><ul><li><ul><li>Monitor information usage over time </li></ul></li></ul><p> 27. Information as a Service Moving From a Project-Based to a Flexible Architecture (SOA) andmore abc DB2 IBMContent Manager Oracle xyz Heterogeneous Applications &amp; Information Insight Information as a Service Optimize, Virtualize, Integrate, Accelerate Master Data, Entity Analytics, Decision Portals, Executive Dashboards, Industry Data Models Extracted or Real-time Standards-based e.g., XQuery, JSR170, JDBC, Web Services... Processes People Tools &amp; Applications Data &amp;Content Business Context Insightful Relationships 28. Information Services for SOA Unprecedented Business Flexibility </p> <ul><li>Store Information </li></ul><ul><li><ul><li>DB2 Viper </li></ul></li></ul><ul><li><ul><li><ul><li>Optimized XML storage </li></ul></li></ul></li></ul><ul><li>Virtualize Information Access </li></ul><ul><li><ul><li>WebSphere Information Server </li></ul></li></ul><ul><li>Integrate Information </li></ul><ul><li><ul><li>WebSphere Information Server </li></ul></li></ul><ul><li>Accelerate Master Information </li></ul><ul><li><ul><li>WebSphere Customer Center </li></ul></li></ul><ul><li><ul><li>WebSphere Product Center </li></ul></li></ul><ul><li><ul><li>IBM Entity Analytics </li></ul></li></ul><ul><li><ul><li>Industry Models </li></ul></li></ul><p> 29. Industry Solutions Deliver Insight On Demand </p> <ul><li>Law Enforcement </li></ul><ul><li><ul><li>Crime Information Warehouse </li></ul></li></ul><ul><li><ul><li>Entity Resolution </li></ul></li></ul><ul><li><ul><li>Anti Money Laundering </li></ul></li></ul><ul><li>Banking </li></ul><ul><li>Basel II and BankingData Warehouse </li></ul><ul><li>Entity Resolution </li></ul><ul><li>Health Care </li></ul><ul><li>Aligned Clinical...</li></ul>