understanding of data warehouse architecture

8/8/2019 Understanding of Data Warehouse Architecture

1/21

Understanding Of DataUnderstanding Of Data

Warehouse ArchitectureWarehouse Architecture


2/21

The Team Foundation reporting warehouseThe Team Foundation reporting warehouseis a traditional data warehouse consisting ofis a traditional data warehouse consisting ofa relational database organized in ana relational database organized in anapproximate star schema and an OLAPapproximate star schema and an OLAPdatabase built on top of the relationaldatabase built on top of the relational

database.database.

Understanding the DataUnderstanding the Data

Warehouse ArchitectureWarehouse Architecture


3/21

HighHigh--level architecturelevel architecture


4/21

Each tool or plugEach tool or plug--in in Team Foundationin in Team Foundationuse a relational database in Microsoft SQLuse a relational database in Microsoft SQLServer 2005 to store the data used by theServer 2005 to store the data used by the

tool in its daytool in its day--toto--day operations. Thisday operations. Thisrelational database is often referred to as therelational database is often referred to as theoperational store.operational store.

Operational StoresOperational Stores


5/21

Common structure databasesCommon structure databases(TfsIntegration and TfsActivityLogging)(TfsIntegration and TfsActivityLogging)

Work item tracking databasesWork item tracking databases(TfsWorkItemTracking and(TfsWorkItemTracking andTfsWorkItemTrackingAttachments)TfsWorkItemTrackingAttachments)

Source control database (TfsVersionControl)Source control database (TfsVersionControl)

Team build database (TfsBuild)Team build database (TfsBuild)

Team test database (TfsBuild)Team test database (TfsBuild)

The operational stores for TeamThe operational stores for Team

Foundation include:Foundation include:


6/21

The warehouse adapter copies and transforms those dataThe warehouse adapter copies and transforms those datafields specified in either the basic warehouse configurationfields specified in either the basic warehouse configurationor in the process template used at the time a new teamor in the process template used at the time a new teamproject is created. If you subsequently change the processproject is created. If you subsequently change the process

template to add or delete which data fields are written totemplate to add or delete which data fields are written tothe data warehouse, these changes are detected the nextthe data warehouse, these changes are detected the nexttime the adapter is run. The adapter runs periodically withtime the adapter is run. The adapter runs periodically witha frequency set by the Run Interval Seconds property. Thea frequency set by the Run Interval Seconds property. Thedefault setting for the refresh frequency is 3,600 seconds,default setting for the refresh frequency is 3,600 seconds,

so give careful consideration to the appropriate refreshso give careful consideration to the appropriate refreshfrequency for your installation.frequency for your installation.

Warehouse AdaptersWarehouse Adapters


7/21

The data in the warehouse areThe data in the warehouse arestored in a set of tables organizedstored in a set of tables organized

in a star schema. The central tablein a star schema. The central tableof the star schema is called the factof the star schema is called the facttable, and the related tablestable, and the related tablesrepresent dimensions. Dimensionsrepresent dimensions. Dimensionsprovide the means forprovide the means fordisaggregating reports into smallerdisaggregating reports into smallerparts.parts.

The Warehouse RelationalThe Warehouse Relational

DatabaseDatabase


8/21

A row in a fact table usually containsA row in a fact table usually containseither the value of a measure or aeither the value of a measure or a

foreign key reference to a dimensionforeign key reference to a dimensiontable. The row represents the currenttable. The row represents the currentstate of every item covered by the factstate of every item covered by the facttable. For example, the Work Item facttable. For example, the Work Item fact

table has one row for every work itemtable has one row for every work itemstored in Work Item operational store.stored in Work Item operational store.


9/21

This allows the data cube toThis allows the data cube toaggregate the measures by day. Theaggregate the measures by day. The

warehouse OLAP data cubewarehouse OLAP data cubeaggregates both data from theaggregates both data from the

underlying star schema and timeunderlying star schema and timedata into multidimensionaldata into multidimensionalstructures.structures.

The Warehouse OLAP CubeThe Warehouse OLAP Cube


10/21

Report Designer is a component of VisualReport Designer is a component of VisualStudio 2005 Team System that allows you toStudio 2005 Team System that allows you todefine the Team Foundation data warehousedefine the Team Foundation data warehouse

as a data source and then design a reportas a data source and then design a reportinteractively. Report Designer providesinteractively. Report Designer providestabbed windows for Data, Layout, andtabbed windows for Data, Layout, andPreview, and you can add datasets toPreview, and you can add datasets toaccommodate a new report design idea, oraccommodate a new report design idea, oradjust report layout based on previewadjust report layout based on previewresults.results.

Report Designer ReportsReport Designer Reports


11/21

Excel ReportsExcel Reports

Team Foundation integrates with MicrosoftTeam Foundation integrates with MicrosoftOffice Excel to allow you to use MicrosoftOffice Excel to allow you to use MicrosoftOffice Excel to manage your project andOffice Excel to manage your project andproduce reports. Microsoft Office Excelproduce reports. Microsoft Office Excelprovides pivot tables and charts for viewingprovides pivot tables and charts for viewingand analyzing multiand analyzing multi--dimensional data. Youdimensional data. You

can bind these pivot tables directly to thecan bind these pivot tables directly to theTeam Foundation OLAP cube, so you canTeam Foundation OLAP cube, so you caninteract with the data in the cube.interact with the data in the cube.


12/21

SecuritySecurity

Security for the Team Foundation dataSecurity for the Team Foundation datawarehouse is defined at the database level,warehouse is defined at the database level,while security for team reports is at the teamwhile security for team reports is at the teamproject level. The Team Foundation Serverproject level. The Team Foundation Serveradministrator determines who has access toadministrator determines who has access tothe data in the data warehouse by grantingthe data in the data warehouse by granting

or revoking permissions on the user'sor revoking permissions on the user'saccount.account.


13/21

Data Warehouse ArchitectureA data warehouse architecture is primarily basedA data warehouse architecture is primarily based

on the business processes of a business enterpriseon the business processes of a business enterprisetaking into consideration the data consolidationtaking into consideration the data consolidation

across the business enterprise with adequateacross the business enterprise with adequatesecurity, data modeling and organization, extent ofsecurity, data modeling and organization, extent ofquery requirements, meta data management andquery requirements, meta data management andapplication, warehouse staging area planning forapplication, warehouse staging area planning for

optimum bandwidth utilization and full technologyoptimum bandwidth utilization and full technologyimplementation.implementation.


14/21

Process Architecture

Describes the number of stages and howDescribes the number of stages and howdata is processed to convert raw /data is processed to convert raw /transactional data into information for endtransactional data into information for enduser usage.user usage.

The data staging process includes threeThe data staging process includes threemain areas of concerns or submain areas of concerns or sub-- processes forprocesses for

planning data warehouse architectureplanning data warehouse architecturenamely Extract, Transform and Load.namely Extract, Transform and Load.


15/21

These interrelated subThese interrelated sub--processes are sometimesprocesses are sometimesreferred to as an ETL process.referred to as an ETL process.

1)1)ExtractExtract-- Since data for the data warehouse can comeSince data for the data warehouse can comefrom different sources and may be of different types, thefrom different sources and may be of different types, theplan to extract the data along with appropriateplan to extract the data along with appropriatecompression and encryption techniques is an importantcompression and encryption techniques is an importantrequirement for consideration.requirement for consideration.

2)2)TransformTransform-- Transformation of data with appropriateTransformation of data with appropriateconversion, aggregation and cleaning besides deconversion, aggregation and cleaning besides de--normalization and surrogate key management is also annormalization and surrogate key management is also animportant process to be planned for building a dataimportant process to be planned for building a data

warehouse.warehouse. 3)3)LoadLoad-- Steps to be considered to load data withSteps to be considered to load data with

optimization by considering the multiple areas where theoptimization by considering the multiple areas where thedata is targeted to be loaded and retrieved is also andata is targeted to be loaded and retrieved is also animportant part of the data warehouse architecture plan.important part of the data warehouse architecture plan.


16/21

Data Model ArchitectureData Model Architecture

In Data Model Architecture (also knownIn Data Model Architecture (also knownasas Dimensional Data ModelDimensional Data Model), there are 3 main data), there are 3 main datamodeling styles for enterprise warehouses:modeling styles for enterprise warehouses:

3rd Normal Form3rd Normal Form -- Top Down Architecture, TopTop Down Architecture, TopDown ImplementationDown Implementation

FederatedFederated Start SchemaStart Schema -- Bottom UpBottom UpArchitecture, Bottom Up ImplementationArchitecture, Bottom Up Implementation

Data VaultData Vault-- Top Down Architecture, Bottom UpTop Down Architecture, Bottom UpImplementationImplementation


17/21

Technology or Technical architectureTechnology or Technical architectureprimary evolved from derivations from theprimary evolved from derivations from theprocess architecture, meta dataprocess architecture, meta datamanagement requirements based onmanagement requirements based onbusiness rules and security levelsbusiness rules and security levelsimplementations and technology toolimplementations and technology tool

specific evaluation.specific evaluation.

Technology ArchitectureTechnology Architecture


18/21

Information ArchitectureInformation Architecture

is the process of translating the informationis the process of translating the informationfrom one form to another in a step by stepfrom one form to another in a step by stepsequence so as to manage the storage,sequence so as to manage the storage,retrieval, modification and deletion of theretrieval, modification and deletion of thedata in the data warehousedata in the data warehouse


19/21

Resource architecture is related to softwareResource architecture is related to softwarearchitecture in that many resources comearchitecture in that many resources comefrom software resources. Resources arefrom software resources. Resources are

important because they help determineimportant because they help determineperformance. Workload is the other part ofperformance. Workload is the other part ofthe equation. If you have enough resourcesthe equation. If you have enough resourcesto complete the workload in the rightto complete the workload in the rightamount of time, then performance will beamount of time, then performance will behigh. If there are not enough resources forhigh. If there are not enough resources forthe workload, then performance will be low.the workload, then performance will be low.

Resource ArchitectureResource Architecture


20/21

The ability to be compliant, consistent andThe ability to be compliant, consistent andrepeatable depends on how the data model is builtrepeatable depends on how the data model is builtunder the covers.under the covers.

There are 3 main data modeling styles forThere are 3 main data modeling styles forenterprise warehouses:enterprise warehouses:

3rd Normal Form3rd Normal Form -- Top Down Architecture, TopTop Down Architecture, TopDown ImplementationDown Implementation

Federated Star SchemasFederated Star Schemas -- Bottom Up Architecture,Bottom Up Architecture,

Bottom Up ImplementationBottom Up ImplementationData VaultData Vault -- Top Down Architecture, Bottom UpTop Down Architecture, Bottom Up

ImplementationImplementation

Various ArchitecturesVarious Architectures


21/21

understanding of data warehouse architecture

Documents