copyright © 1998-2014 curt hill foundational software systems a brief overview

Copyright © 1998-2014 Curt Hill

Foundational Software Systems

A Brief Overview

Introduction

• Certain classes of software need to be understood by all technical people

• Operating Systems• Servers• Databases and Data Warehouses• Enterprise Resource Planning• Customer Relationship

ManagementCopyright © 1998-2014 Curt Hill

Outroduction

• What this presentation is not about:

• Virtualization• Cloud computing • Service Oriented Architecture• Software as a Service• These are important topics to be

considered at another time


OS

• Foundation of the foundation• Every general purpose computer

has one– Many special purpose computers as

well

• Applications are why we buy the computer, but the operating system is required to run the applications


What does an OS do?

• Execute application programs• Provide services to these programs• Establish and manage a file system• Utility functions• Many functions that are used by

most programs will eventually be incorporated into the OS


Execute application programs

• Interpret commands– Interpret typed commands or mouse

actions

• Load and execute programs• Provide services to the running

programs– The exact description of these

services distinguishes different OSs


Provide services to the program

• Memory management– Memory allocation – Virtual or fixed memory

• I/O management– Manage file activity– Often the most visible and important

• Task Scheduler– Inter-process communication– Managing priorities


Establish and manage a file system

• Determine how things are stored on disk– Do we have a directory or just search?– Are things stored in a contiguous fashion

or scattered?– How are things named?– How are things accessed?

• Interrupt processing• Utility functions

– Prepare a disk for use– Copy/move/delete files– Show contents of a file or disk


How are files accessed?

• Access to files is a program service– Program says that it wants access to a file

of some filename– System finds it– Gets it ready– Gives it to the program one chunk at a

time

• Several things should be transparent to program

• Buffering• Devices


Utility functions• Protection

– Files from deletion by someone other than owner

– Memory of one program from another• Hardware manipulation

– Mount disks and tapes• Miscellaneous

– Error detection– Multitasking– Accounting


Pieces of the operating system

• CPU management• Memory management• Process management• I/O management• User management• File management• Others


OS Layers

• Most OSs are onion shaped– For historical and maintenance reasons– Historically the innermost was all that

was available

• Each layer then communicates with adjacent layers through well defined paths– Well defined in terms of this particular

OS

• Layers: Kernel, Service, Command


Kernel is the innermost layer• Low level activities only

– Load and/or execute a program– Device allocation and access– Process management– Low level file access

• This is the lowest level interface to the hardware and the only one that touches the hardware directly

• In DOS there is the BIOS which amounts to a kernel

• The kernel is usually always resident in the memory

• Basic process management is usually here, since we need to make each program we load a process


Service layer• Set of routines that provide services

– Use the kernel– This is where the traditional file manipulation

commands are executed, such as opening and closing files, reading and writing records

– Entire file copying is usually done at the next higher level

• The service layer represents the services that are available to a program

• The service layer is also designed to generalize the interface between devices– A CRT, disk and tape are wildly different, yet we

would like their interface to look similar


Service Layer

• In the assembly language interface each of the service calls becomes a macro or subroutine call

• In a HLL it is usually built into the command part of the language or is in a library of some sort

• Some process management is here as well– Inter-process communication


Command layer• The interface that allows the user to

interface with the system directly• This includes command interpretation• This is Command.com in DOS or

program manager in Windows• Many utility functions are placed in this

layer– The DOS Format is a command in this layer

• It performs an essential function, but is its own program


Servers• Client server computing is the basis of

the Internet – Among other things

• Requires two computers– A client – A server

• Communication medium• The illusion is that each client

computer thinks it is has exclusive access to the server


The Paradigm

• Server provides one service• Waits for requests for that service• Client initiates a request• Server replies to the request• Information may flow in both

directions• Many clients may make requests

with just the one server


Why Client - Server?

• Specialization– The client handles the user interface

using local computational ability– The server manages the data and

communication

• Sharing– One server may support many clients– Services provided are effectively shared– Data may be maintained in only one

location


Clients

• An application program that runs on the local computer

• Able to use the local computer for doing work

• When invoked by a user, it initiates communication with a server

• Handles all the user interface issues

• Only used for a single session


Example• The ubiquitous client is the web

browser• Several functions:• Communicate with server• Display HTML/XML• Interpret client-side languages, such as

JavaScript• The trend is for the web browser to be

the universal interface to every kind of server on the web


Servers• Specialized program that typically

provides only one service• May handle multiple client requests

simultaneously (or pseudo simultaneously)

• Interacts with users in many sessions– Both simultaneously and over long time

periods

• Waits for a request and then produces the reply


Example

• The web server• Waits for requests using Hyper Text

Transmission Protocol• Produces results in HTML/XML• May handle many clients

simultaneously


Connection Diagrammed


Server

Client

RequestReply

Client Client

Request

ReplyReplyRequest

Interaction Diagrammed

• The server is not waiting if it is processing other requests

• The client may also perform other processing while waiting for the reply


Client

ServerWait Wait

Wait

Request Reply

Time

Processing

Multiple tiers• The classic client-server is a two tier

system• The client is one tier and the server

another• Three or more tiers are also possible• The middle server functions as a

– Server to the client– Client to other servers– Known as middleware

• Often used to distribute load


A Three Tier Approach


Server

Application Server

Client ClientClientClient

ServerServer Server

Examples of Multi-Tier

• Authentication server– Maintains a connection to verify user– Funnels all requests to appropriate

server

• Shopping cart– Records purchases– The product lookup/purchase handled

by other servers

• Distributed Database Management System


Horizontal Distribution• Multi-Tier is a vertical distribution• A Horizontal distribution may also be

effective• A single server owns the web address:

www.x.com• It then distributes all further requests

to a rotation of identical web servers– www1.x.com– www2.x.com– …– wwwN.x.com


http://www.x.com/

Horizontal Distribution


www.x www1.x wwwN.xwww2.x …

Client ClientClientClientClientClient

FirstRequest Subsequent

Request

Database

• The web server may be the most common server type, but the database server is the workhorse

• It is the foundation for many other important classes of programs


Historical Perspective• Actions we might want on any file of

fixed records:– Create– Update (add, remove, change records)– Sort– Generate any of several reports

• Each of these would be a program for an overworked programming staff– Typically a COBOL program


Database

• The key– All these programs are about the

same– All that changes is the underlying file

• The solution– Describe the file in a general way

• This is the metadata – data that describes data

– Generate a program that handles the file based on the description


How to describe a file• A file is a collection of records• Each record is a collection of fields

– Typically only one type of record in a file

• Each field is described by a:– Name– Type

• For example numeric, string, boolean etc.

– Length• Booleans have a predefined length, others

require specification


Database Advantages 1

• Data independence– Application program no longer need

some or all of the files– Do not know or care how data is

stored, aka abstraction – Simplifies application development

• Efficient access– The DBMS employs sophisticated

access techniques seldom used by normal programmers



• Integrity constraints– The DBMS may check data in a way

seldom done in normal file processing– Eg. Account validity

• Security– A DBMS may enforce requirements on

who can access the data and in what way



• Administration– Minimize redundancy– Manage sharing of the data– Optimize for the enterprise, not a

small group– Easier to backup the data

• Concurrent access– Manages the simultaneous update

problem


Functions of a Database

• Data storage and retrieval• User-accessible catalog• Transaction support• Concurrency support• Recovery services• Authorization services• Support for data communication• Integrity services• Data independence services


Data storage and retrieval

• While hiding the file structure, users may: – Add– Delete– Update in place

• Most relational databases support a common language– Structured Query Language


SQL

• A language where a question is asked of the database and it provides an answer:– Pronounced Sequel or spelled out– Usually state what is wanted rather

than how to get it

• The Data Manipulation Language usually includes both query and update facilities


Transaction Support

• The unit of update is a transaction– All or nothing– No partial updates are allowed

• The DBMS must guarantee the integrity of a transaction, even in the face of:– Program errors– Incorrect input– System crashes


Transactions: ACID• Atomic

– Transaction perceived to be indivisible• Consistent

– Transforms database from one consistent state to another

• Isolated– Understandable without regards to any

other agents

• Durable– Once committed permanence is

guaranteed even with system crashes


Types of database• A large organization often have

two distinct types of databases• A transactional or operational

database• An analytical database or data

warehouse• NoSQL• These have separate uses


Database uses• The database is about transactions

– Maintaining current knowledge

• The data warehouse is about decision support– Spotting trends

• NoSQL is much harder to classify, any of these– Massive data– Unique organizations– Distributed – Among many others


Characteristics

• Operational database have:– Strict performance requirements– Predictable workloads– Small units of work– High utilization

• Data warehouses are contrary in every respect


Warehouses

• A data warehouse– Subject oriented, integrated, non-

volatile, time-variant collection of data in support of management decsions

• Data warehouses support– OLAP (OnLine Analytical Processing)– DSS/EIS (Decision Support Systems or

Executive Information Systems)– Data mining


What is Data Mining?• Exploratory data analysis based on a

data warehouse– Knowledge Discovery in Databases (KDD)

• Data Mining extracts previously unknown and potentially useful information– Rules, constraints, correlations, patterns,

signatures and irregularities

• The goal is to automate the methods for finding these in the data


Goals of Data Mining

• Prediction of future behaviors– Seasonal or non-seasonal trends– How will consumers respond to

discounts?– Allows the enterprise to be ready

• Identification of item, event or activity– Intruders may be identified by the

files they access or programs they use


Goals Again

• Classification of categories of users or products– Shoppers may be categorized as:

• Discount seeking• Rush• Regular• Attached to certain brand names

– The store may be made more friendly to such

• Optimize the use of time, space, materials and money


Knowledge Discovery

• There are several types of discoverable knowledge:

• Association Rules• Classification hierarchies• Sequential patterns• Time series patterns• Clustering


Databases• The two types of databases are the

transactional and analytical– The day to day transactional feeds the

analytical

• We do not want knowledge discovery, data mining and the like to occur on the transactional database– Could seriously impede performance

• Either might be used in the next two applications– Transactional is more likely


Some Definitions• Data

– Small measurable pieces of information– Foundation of knowledge

• Information– Organized data– Putting data into meaningful patterns

• Knowledge – A higher aggregation of information and

data that enables interpretation– Ability to use information– Sets of rules and relationships


Example• Any numbers or text is data

– Such as: 324 3.83

• It becomes information when we know its context– Such as these are the number of items

in stock and how much we paid per unit

• Knowledge is when we take this information and combine it with processes and rules– Computing our price for selling

something made out of these


Data Mart

• A data warehouse restricted to a single subject– Such as a group or line of business

within a larger organization

• A dependent data mart obtains its data from the central organizational data warehouse

• An independent data mart obtains data from other sources– No overlap with the data warehouse


Knowledge Warehouse

• An architectural integration of knowledge management, decision support, artificial intelligence and data warehousing

• Built on top of a data warehouse• Must also encode the rules and

processes of the business– These rules used to only be in the

minds of employees– Next captured on documents


ERP• Enterprise Resource Planning• The key idea behind database

technology is that the process of manipulating files is basically the same– The only thing that changes is the file

format

• The key to ERP is that many business processes are very similar– Payroll is just payroll

• Took off because of the Euro and Y2K threat


ERP Areas

• Financial Accounting• Management Accounting• Human Resources• Manufacturing• Supply Chain Management• Project Management• Customer Relationship Management• These were all separate programs at

one time


Configuration

• Most systems need to be customized by programmers modifying the code

• Most ERPs may be configured to customize to a particular situation

• This configuration is usually much easier than code modification

• The program typically handles best practices, but the configuration can take into account the idiosyncrasies of this business


Simple Example

• The form of a product number drastically varies between companies– Usually groups of letters and digits

• Configuring the system to store and validate the type used by a particular company needs to be a simple configuration


CRM• Customer Relationship Management• The notion of a call center• A past, present or future customer calls• A representative, with no previous

knowledge of this customer, has to deal with them

• CRM puts all the information necessary to help in front of the representative, regardless of the purpose of the call


Types of calls

• Sales– Availability, delivery, costs and

discounts

• Support after the sale– Warranty, replacements, returns

• Information and marketing– Request for information

• Appointments for sales force• Analysis of customer or products


Implementation

• The leading CRMs, like ERPs are highly configurable

• Also like ERPs they are backed by a transactional database

• Both generate large quantities of SQL on the fly

• ERPs are starting to absorb CRMs


Finally

• The type or size of businesses that would be architected will likely have all of these

• They will also have serious customizations to these products

• Many other specialized applications also exist– EG. EA tools


copyright © 1998-2014 curt hill foundational software systems a brief overview

Documents