it ready dw_day3_rev00
DESCRIPTION
TRANSCRIPT
![Page 1: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/1.jpg)
Northern Region IT Professional Development Program 2010
Data Warehousing (DAY 3)
Siwawong W.Project Manager
2010.05.26
![Page 2: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/2.jpg)
Northern Region IT Professional Development Program 2010
Agenda
09:00 – 09:15 Registration
09:15 – 09:30 Review 2nd Day class
09:30 – 10:00 Building Business Intelligence
10:00 – 10:30 Introduction to SSIS
10:30 – 10:45 Break & Morning Refreshment
10:45 – 12:00 SSIS Workshop & Exercise
12:00 – 13:00 Lunch Break
13:00 – 15:00 Introduction to SSAS
15:00 – 15:15 Break
15:15 – 16:00 SSAS Workshop & Exercise
![Page 3: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/3.jpg)
Northern Region IT Professional Development Program 2010
2nd Day Review
![Page 4: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/4.jpg)
Northern Region IT Professional Development Program 2010
Dare Warehouse: Architecture Review
• Loading (aka, ETL)
• Refreshing: When & How?
• Structure/Modeling: star vs snow-flake schema.
• Data Marts
• Query Processing– Indexing: Bitmap vs Join– Pre-Computed Aggregates– SQL Extension
![Page 5: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/5.jpg)
Northern Region IT Professional Development Program 2010
OLAP: Review
• ROLAP vs MOLAP
• Slicing & Dicing
• Successful vs Pitfalls
![Page 6: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/6.jpg)
Northern Region IT Professional Development Program 2010
Building Business IntelligenceWith MS-SQL server 2005
![Page 7: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/7.jpg)
Northern Region IT Professional Development Program 2010
What’s Business Intelligence (BI)?
• U - sed in spotting, digging out, and analyzing business data
• provide historical, current, and predictive views of business operations.
• Common functions of Business Intelligence technologies are– reporting– online analytical processing– Analytics– data mining– etc.
Refer from http://en.wikipedia.org/wiki/Business_intelligence
![Page 8: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/8.jpg)
Northern Region IT Professional Development Program 2010
What’s Business Intelligence (BI)?
Business intelligence (BI) is more of a concept than a single technology. The goal is to gain insight into the business by bringing together data, formatting it in a way that enables better analysis, and then providing tools that give users power—not just to examine and explore the data, but to quickly understand it.
The above definition from Business Intelligence with Microsoft Office PerformancePoint Server 2007
![Page 9: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/9.jpg)
Northern Region IT Professional Development Program 2010
Related to Data Warehouse
Data Warehouse = Business Intelligence
To me DW and BI are the same thing. A data warehouse is useless if users cannot access the data in an easy manor. Therefore, for this presentation these two words are interchangeable.
![Page 10: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/10.jpg)
Northern Region IT Professional Development Program 2010
Why Business Intelligence?
Poor Visibility and Poor Visibility and Reaction to Reaction to
Market EventsMarket Events
High Business and IT High Business and IT Operation CostsOperation Costs
Poor Understanding of Poor Understanding of Customer NeedsCustomer Needs
Inefficient Supply Inefficient Supply Chains and Demand Chains and Demand
ChainsChains
Poor Business Poor Business PerformancePerformance
Management by Management by SpreadsheetsSpreadsheets
Data Privacy Concerns and Data Privacy Concerns and Information OverloadInformation Overload
Compliance (BASEL, Compliance (BASEL, Sarbanes Oxley) Sarbanes Oxley)
Today’s informationToday’s informationTECHNOLOGYTECHNOLOGY
80% of IS Budget to 80% of IS Budget to ‘business as usual’‘business as usual’
![Page 11: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/11.jpg)
Northern Region IT Professional Development Program 2010
What Products Are Involved?
• Database Technologies– MS-SQL Server 2005 (Database Engine)– MS-SQL Server Analysis Services (SSAS)– MS-SQL Server Integration Services (SSIS)
• User Interface Technologies– MS-SQL Server Reporting Services (SSRS)– MS-SQL Server Management Studio (SSMS)– Excel
![Page 12: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/12.jpg)
Northern Region IT Professional Development Program 2010
OLAP Leadership
http://www.olapreport.com
![Page 13: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/13.jpg)
Northern Region IT Professional Development Program 2010
BI Platform Selection Requirements
• Focus on operational BI• Extending the reach of BI• Scope of functionality• Scalability• Availability• Simplicity
![Page 14: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/14.jpg)
Northern Region IT Professional Development Program 2010
MS-SQL SERVER 2005 BI
Analysis ServicesAnalysis ServicesOLAP & Data MiningOLAP & Data Mining
Data TransformationData TransformationServicesServices
SQL ServerSQL ServerRelational EngineRelational Engine
Reporting ServicesReporting Services Managem
ent ToolsM
anagement ToolsD
evel
opm
ent T
ools
Dev
elop
men
t Too
ls
Refer from: http://www.renaissance.co.il/ivbug/meeting74/SQL%20Server2005%204%20VB%20group.ppt
![Page 15: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/15.jpg)
Northern Region IT Professional Development Program 2010
Analysts
Information Consumers
Information Explorers
5-10% of users
15-25% of users
65-80% of usersReporting Services
Analysis Services
BI vs Users
![Page 16: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/16.jpg)
Northern Region IT Professional Development Program 2010
Microsoft BI Components
Component SQL Server 2000 SQL Server 2005
Extraction, Transformation and Loading
SQL Server 2000 Data Transformation Services (DTS)
SQL Server 2005 Integration Services
Relational Data Warehouse SQL relational database SQL relational database
Multidimensional Database SQL Server 2000 Analysis Services SQL Server 2005 Analysis Services
Data Mining SQL Server 2000 Analysis Services SQL Server 2005 Analysis Services
Managed Reporting SQL Server 2000 Reporting Services SQL Server 2005 Reporting Services
Ad hoc Query and Analysis Microsoft Office Products SQL Server 2005 Report BuilderBusiness Scorecard ManagerMicrosoft Office Products
Database Development Tools
SQL Server 2000 Enterprise Manager, Analysis Manager, Query Manager…
SQL Server 2005 Business Intelligence Development Studio
Database Management Tools
SQL Server 2000 Enterprise Manager, Analysis Manager
SQL Server 2005 Management Studio
![Page 17: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/17.jpg)
Northern Region IT Professional Development Program 2010
Business Intelligence Opportunity
Data acquisition Data acquisition from source from source systems and systems and integrationintegrationData Data transformation transformation and synthesisand synthesis
Data enrichment, Data enrichment, with business with business logic, hierarchical logic, hierarchical viewsviewsData discovery Data discovery via data miningvia data mining
Data presentation Data presentation and distributionand distributionData access for Data access for the massesthe masses
IntegrateIntegrate AnalyzeAnalyze ReportReport
![Page 18: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/18.jpg)
Northern Region IT Professional Development Program 2010
Information Information DeliveryDelivery
Data MartsData Marts
Data AnalysisData Analysis(OLAP, Data(OLAP, Data
Mining)Mining)
Familiar,Familiar,PowerfulPowerfulBI ToolsBI Tools
ClientClient PortalPortal
DevicesDevicesEnterpriseEnterpriseETLETL
Third Party Third Party ApplicationsApplications
EnterpriseEnterpriseReportingReporting
Performance Performance ScorecardScorecard
Interactive Interactive ReportsReports
Business Business InsightsInsights
DataDataWarehouseWarehouse
Tightly integrated “all-in-one” technology solution… Tightly integrated “all-in-one” technology solution… increased manageability and the best economicsincreased manageability and the best economics
Highly intuitive, visual tools…Highly intuitive, visual tools…greater productivity from greater productivity from
developers to usersdevelopers to usersComprehensive ability to integrate any data… Comprehensive ability to integrate any data…
improved data completeness improved data completeness
CRMCRM
LOBLOB
ERPERP
SourceSourceSystemsSystems
![Page 19: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/19.jpg)
Northern Region IT Professional Development Program 2010
SQL Server Integration Services (SSIS)
![Page 20: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/20.jpg)
Northern Region IT Professional Development Program 2010
MS-SQL2005: SSIS
• Introduction to SSIS• The Import and Export Wizard• Creating A Package• Components of Package• Saving & Running Packages
![Page 21: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/21.jpg)
Northern Region IT Professional Development Program 2010
Introduction to SSIS?
• A feature of SQL Server 2005
• Latest incarnation of Data Transformation Services (DTS)
• Used to transform and move data into and out of files and databases
• SSIS is a p latform for building high performance data integration sol utions, including extraction, transformation, and load (ETL) package
s for data warehousing
• SSIS provides a way to build packages made up of tasks that can move data around from place to place and alter it on the way
![Page 22: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/22.jpg)
Northern Region IT Professional Development Program 2010
DTS today – a little history
• DTS - SQL Server 7.0– “Visual BCP” – a useful utility
• DTS - SQL Server 2000– Easy (but slow) workflow & transform engine– Customizable
• SSIS – SQL Server 2005– A completely new codebase– Enterprise class ETL– Exceptional BI integration – and more– Rich APIs and extensibility
![Page 23: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/23.jpg)
Northern Region IT Professional Development Program 2010
SSIS Components
• SSIS has two broad groups of components– Server-Side
Extensions to the DBMS that enable advanced SSIS tasks (simpler tasks are supported on most DBMSs via standard drivers and SQL commands)
Extensions are “invisible” to the user/programmer unless they’re absent or incorrectly installed
– Client-Side Software components for both low-level and high-level integration tasks Interfaces to data sources (e.g., spreadsheets, text files, and various DBMSs) Interfaces to data destinations A toolkit (Visual Studio 2005 plus required components and templates) that
enables users/programmers to combine components and interfaces to accomplish specific high-level tasks
![Page 24: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/24.jpg)
Northern Region IT Professional Development Program 2010
SSIS Architecture
![Page 25: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/25.jpg)
Northern Region IT Professional Development Program 2010
Development Environment
• Visual Studio (Business Intelligence Development Studio)
– aka: BIDS
• Visual Studio 2005, .NET Framework 2.0• SQL Server Integration Services components need to be
installed in both Visual Studio and SQL Server
![Page 26: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/26.jpg)
Northern Region IT Professional Development Program 2010
Import & Export Wizard
• Though SSIS is almost infinitely customizable, Microsoft has produced a simple wizard to handle some of the most common ETL tasks: importing data to or exporting data from a SQL Server database. The Import and Export Wizard protects you from the complexity of SSIS while allowing you to move data between any of these data sources:
– SQL Server databases– Flat files– Microsoft Access databases– Microsoft Excel worksheets– Other OLE DB providers
![Page 27: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/27.jpg)
Northern Region IT Professional Development Program 2010
Demo Import & Export Wizard
New DB
(1) Start on New DB
(2) Selected Data Source (Input)
Select “AdventureWorks”
(3) Selected Destination (Output)
Default destination is selected DB
![Page 28: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/28.jpg)
Northern Region IT Professional Development Program 2010
Demo Import & Export Wizard (Cont’)
(4) Select Option for import
HumanResources.Department HumanResources.EmployeeHumanResources.EmployeeAddress HumanResources.EmployeeDepartmentHistoryHumanResources.EmployeePayHistory
(5) Selected tables
(6) Edit mapping columns
![Page 29: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/29.jpg)
Northern Region IT Professional Development Program 2010
Demo Import & Export Wizard (Cont’)
(7) Save & Excuse
(8) Confirm before start
(9) Result Status
![Page 30: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/30.jpg)
Northern Region IT Professional Development Program 2010
Creating A Package
• The Import and Export Wizard is easy to use, but it only taps a small part of the functionality of SSIS.
• To really appreciate the full power of SSIS, you’ll need to use BIDS to build an SSIS package
• Package is a collection of SSIS objects including:– Connections to data sources.– Data flows, which include the sources and destinations that extract
and load data, the transformations that modify and extend data, and the paths that link sources, transformations, and destinations.
– Control flows, which include tasks and containers that execute when the package runs. You can organize tasks in sequences and in loops.
– Event handlers, which are workflows that runs in response to the events raised by a package, task, or container.
![Page 31: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/31.jpg)
Northern Region IT Professional Development Program 2010
Demo: Creating A Package
(1) Click here
(2) File New Project…
(3) BI Projects Integration Services Project
Selected Here
![Page 32: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/32.jpg)
Northern Region IT Professional Development Program 2010
Components of Package
Main panel
• Control flows• Data flows• Event handlers• Variables, expressions, package configurations• Connection managers
![Page 33: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/33.jpg)
Northern Region IT Professional Development Program 2010
Connections
• Practically any data source can be used
• Data Source Views– Allows multiple sources to be viewed as a single logical source– Reusable and can use friendly names– Static – underlying structure is not updated
![Page 34: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/34.jpg)
Northern Region IT Professional Development Program 2010
Working with Connection Managers
• Control and data flow objects may require a connection manager
• Various types (OLE DB, flat file, ADO, Excel, OLE DB, FTP)
• Available properties depend on type• All have a connection string• Browse to create a connection string
![Page 35: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/35.jpg)
Northern Region IT Professional Development Program 2010
Demo for Connection Manager
1) Right click in this area
2) Select OLE DB Connection
3) Show Existing Connection
4) Create new connection
5) Select SQL Native Client provider
6) Configuration for link to new DB
6) Test Connection for confirm
![Page 36: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/36.jpg)
Northern Region IT Professional Development Program 2010
Demo for Connection Manager
7) Finally, we get new connection. Select and press OK
8) Then, we get new connection
9) Create New data source is “Flat File” (we use department.txt for sample)
11) Select department.txt (sample)
12) Check it!
10) Input name & information
![Page 37: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/37.jpg)
Northern Region IT Professional Development Program 2010
Demo for Connection Manager
13) Click on Column
14) Verify data in text file
15) Click on Advance icon for go to Advance Page
![Page 38: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/38.jpg)
Northern Region IT Professional Development Program 2010
Demo for Connection Manager
16) Click on New button
17) Display new column
18) Change column name to DepartmentName
19) Click on OK button
20) New Connection appear
![Page 39: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/39.jpg)
Northern Region IT Professional Development Program 2010
Demo for Connection Manager
21) Right-Click on DepartmentList (new Connection)
23) Select on new connection and check in Properties panel
22) Select copy & paste on connection manager
24) Change values: - Change the Name property to DepartmentListBackup. Change the ConnectionString property to C:\DepartmentsBackup.txt.
25) Finally, we get..
![Page 40: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/40.jpg)
Northern Region IT Professional Development Program 2010
Building Control Flows
• File system tasks (copy, delete, rename files, FTP)• Execute tasks (SQL Stored Procedure, Windows
process task , SSIS package task)• Control structures (For loop container, for each loop
container)• Data flow tasks
![Page 41: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/41.jpg)
Northern Region IT Professional Development Program 2010
Control Flow Objects
• Tasks are things that SSIS can do, such as execute SQL statements or transfer objects from one SQL Server to another.
• Maintenance Plan tasks are a special group of tasks that handle jobs such as checking database integrity and rebuilding indexes.
• The Data Flow Task is a general purpose task for ETL (extract, transform, and load) operations on data. There’s a separate design tab for building the details of a Data Flow Task.
• Containers are objects that can hold a group of tasks.
![Page 42: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/42.jpg)
Northern Region IT Professional Development Program 2010
SSIS Container
• For Loop: Repeat a task a fixed number of times• Foreach : Repeat a task by enumerating over a group of
objects• Sequence: Group multiple tasks into a single unit for
easier management
![Page 43: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/43.jpg)
Northern Region IT Professional Development Program 2010
View - Control Flow
• Process steps or events
• Steps are linked together by precedence constraints– Value – success (green), failure (red), completion (blue)– Evaluation operation – constraint, expression– Multiple constraints - logical AND, OR
![Page 44: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/44.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building Control Flow
1) Pin the tool bar
2) Select File System Task
3) Drag to area of Control Flow
4) Drag to area of Control Flow
5) Make Green arrow connect
![Page 45: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/45.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building Control Flow
6) Double-click on link.
7) Change the Value from Success to Completion, because you want the Data Flow Task to execute whether the File System Task succeeds or not.
8) Click OK
![Page 46: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/46.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building Control Flow
9) Double-Click on this task
10) Change PropertySet the Source property toDepartmentList. Set the Destination property to DepartmentListBackup.Set the OverwriteDestinationFile property to True.
11) Click on OK button
![Page 47: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/47.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building Control Flow
12) Run it!!!
All are Green! But it’s warning in output dialog
Overall status
Check result in folder, new file has created!
![Page 48: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/48.jpg)
Northern Region IT Professional Development Program 2010
Building the Data Flows
• Data Flow Sources & Destinations• Data Flow Transformations
![Page 49: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/49.jpg)
Northern Region IT Professional Development Program 2010
Data Flow Sources & Destinations
• Data is strongly typed• Editor – connection manager, columns, error handling• Advanced editor – column mappings, data type
properties, error handling• Input and output column mappings are generated
automatically• Error handling can be defined for individual column and
error type• Errors can be directed to specific output files
![Page 50: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/50.jpg)
Northern Region IT Professional Development Program 2010
Data Flow Transformations
• Derived Column Transformation Editor useful for adding new columns or changing data type
• Original and derived columns• Expression builder
![Page 51: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/51.jpg)
Northern Region IT Professional Development Program 2010
Transformations/Effected
• Aggregate: Aggregates data from transform or source.• Character Map: This transformation makes string data changes for you,
such as changing data from lowercase to uppercase.• Data Conversion: Converts a column's data type to another data type.• Data Mining Query: Performs a data-mining query against Analysis
Services.• Fuzzy Grouping: Performs data cleansing by finding rows that are likely
duplicates.• Fuzzy Lookup: Matches and standardizes data based on fuzzy logic. For
example, this can transform the name Jon to John.• Merge Join: Merges two data sets into a single data set using a join
function.• OLE DB Command: Executes an OLE DB command for each row in the
data flow.• etc.
![Page 52: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/52.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building the Data Flows
1) Select Data Flow tab
2) Select Data Flow task3) Drag & Drop
OLE Source
4) Drag & Drop Character Map
5) Drag & Drop File Destination
6) Drag green Arrow to below box
![Page 53: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/53.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building the Data Flows
7) Double-click on OLEDB source
8) Select table/views: HumanResources.Department
9) Click on OK button
![Page 54: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/54.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building the Data Flows
10) Double-click on Character Map
11) Select Name column
12) Change to in-place change 13) Change to
Upper Case
Then press OK button
![Page 55: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/55.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building the Data Flows
14) Double-click on Flat File Destination
15) Select the DepartmentList Flat File Connection Manager
16) Drag Name column to DepartmentName column
Then press OK button
![Page 56: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/56.jpg)
Northern Region IT Professional Development Program 2010
Demo for Building the Data Flows
17) Run it!!!
All are Green!
Overall status
Result: transform one of the columns in that table to all uppercase characters, and then write that transformed column out to a flat file.
![Page 57: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/57.jpg)
Northern Region IT Professional Development Program 2010
Creating Event Handlers
• Event handler tasks can be defined for each executable• Events include OnPostvalidate, OnTaskFailed,
OnVariableValueChanged• An error handler is itself a control flow and can include
multiple steps
![Page 58: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/58.jpg)
Northern Region IT Professional Development Program 2010
Demo for Creating Event Handlers
1) Create new table on our Test DB CREATE TABLE DepartmentExports(
ExportID int IDENTITY(1,1) NOT NULL,ExportTime datetime NOT NULLCONSTRAINT DF_DepartmentExports_ExportTime DEFAULT(GETDATE()),
CONSTRAINT PK_DepartmentExports PRIMARY KEY CLUSTERED( ExportID ASC)
)
2) Click on Event Handler
3) Click on Data Flow Task
4) Select OnPostExecute
5) Click on this link
![Page 59: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/59.jpg)
Northern Region IT Professional Development Program 2010
Demo for Creating Event Handlers
1) Click on Execute SQL task
2) Drag and drop in this area
3) Double-click on this object
4) Change ConnectType = OLEDB
5) Select OLEDB connection
6) Input SQL statementINSERT INTO DepartmentExports (ExportTime) VALUES (GETDATE())
7) Click on OK button
![Page 60: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/60.jpg)
Northern Region IT Professional Development Program 2010
Demo for Creating Event Handlers
8) Run it!!! (Please delete the existing files and use original files before run)
All are Green!
Overall status
Check result after running the package
![Page 61: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/61.jpg)
Northern Region IT Professional Development Program 2010
Execution Results (in BIDS)
See Progress tab (after running)
Progress is OK
Warning message
![Page 62: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/62.jpg)
Northern Region IT Professional Development Program 2010
Saving and Running Packages
• When you work in BIDS, your SSIS package is saved as an XML file (with the extension dtsx) directly in the normal Windows file system
• Storing SSIS packages in the Package Store or the msdb database makes it easier to access and manage them from SQL Server’s administrative and command-line tools without needing to have any knowledge of the physical layout of the server’s hard drive.
![Page 63: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/63.jpg)
Northern Region IT Professional Development Program 2010
Package Features in SQL Server
• Packages must be imported and re-imported if changed– Even though they are displayed when copied to the package
directory
• Runtime Options– Packages lack configuration files – these can be added at run
time– Command files can be specified– Connection manager connection strings– Other options
![Page 64: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/64.jpg)
Northern Region IT Professional Development Program 2010
Demo for Saving Packages
(1) Save current package as…
(2) Input information as below picture:
(3) Specifield path as /File System/ExportDepartments
(4) Click on OK button
![Page 65: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/65.jpg)
Northern Region IT Professional Development Program 2010
Demo for Running Packages
(1) Select Connect Integration Services
(2) Input login information and press Connect.
(3) Expand & Find store package
(4) Select Run Package
![Page 66: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/66.jpg)
Northern Region IT Professional Development Program 2010
Demo for Running Packages
(5) Click on Execute
(6) Display result on pop-up window
![Page 67: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/67.jpg)
Northern Region IT Professional Development Program 2010
SSIS: Exercise
• One common use of SSIS is in data warehousing - collecting data from a variety of different sources into a single database that can be used for unified reporting. In this exercise you’ll use SSIS to perform a simple data warehousing task.
• Use SSIS to create a text file, EmployeeList.txt, containing the last names and network logins of the AdventureWorksAdventureWorks employees.
– Retrieve the last names from the Person.Contact table in the AdventureWorksAdventureWorks database.
– Retrieve the logins from the HumanResources.Employee table in the Sample database.
• You can use the Merge Join data flow transformation to join data from two sources.• One tip: the inputs to this transformation need to be sorted on the joining column.
![Page 68: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/68.jpg)
Northern Region IT Professional Development Program 2010
SQL Server Analysis Services (SSAS)
![Page 69: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/69.jpg)
Northern Region IT Professional Development Program 2010
MS-SQL2005: SSAS
• Introduction MS-SQL2005 SSAS• Understanding Analysis Services• Creating a Data Cube• Exploring a Data Cube
![Page 70: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/70.jpg)
Northern Region IT Professional Development Program 2010
Introduction to MS-SQL2005 SSAS
![Page 71: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/71.jpg)
Northern Region IT Professional Development Program 2010
Analysis ServicesWhy OLAP and Data Mining Matter
• Powerful business information modeling• Cross platform data integration• Integrated Relational & OLAP views• The best of MOLAP to ROLAP• Data enrichment and advanced analytics• Key Performance Indicators and
Perspectives• Real-time, high performance
– Real-time data in OLAP Cubes– Very fast and flexible analytics
• XML standards for Data Access and Web Services integration
• Cost and time savings for customers integrating with other systems
![Page 72: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/72.jpg)
Northern Region IT Professional Development Program 2010
Dashboards
Rich Reports
BI Front Ends
Spreadsheets
Ad Hoc Reports
AnalysisServices
CacheCacheX
ML/
A o
r OD
BO
XM
L/A
or O
DB
O
UDMUDM
SQL SQL ServerServer
TeradataTeradata
OracleOracleDB2DB2 LOBLOB
DWDW
DatamartDatamart
Analysis ServicesHigh-level Architecture
![Page 73: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/73.jpg)
Northern Region IT Professional Development Program 2010
• Business Intelligence Enhancements
– Auto generation of time and other dimensions based on type
– KPIs, MDX scripts, translations, currency…
• Data Mining– 10 Mining Algorithms– Smart applications
• XML standards for Data Access & Web services integration
– $$ saving for customers integrating our solution with other systems
• Unified Dimensional Model– Powerful business information
modeling– Cross platform data integration– Integrated Relational & OLAP
views– KPIs & Perspectives
• Proactive caching– Real-time data in OLAP Cubes– Very fast and flexible analytics
SQL Server Analysis ServicesNew Paradigm for the Analytics Platform
![Page 74: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/74.jpg)
Northern Region IT Professional Development Program 2010
Understanding Analysis Services
• Cube• Dimension table• Dimension• Level• Fact table• Measure• Schema
![Page 75: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/75.jpg)
Northern Region IT Professional Development Program 2010
Cube
• A collection of data that’s been aggregated to allow queries to return data quickly
• Cubes are ordered into dimensions and measures.• Dimensions come from dimension tables, while
measures come from fact tables.
![Page 76: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/76.jpg)
Northern Region IT Professional Development Program 2010
Dimension table & Dimension
• Dimension Table: Contains hierarchical data by which you’d like to summarize
• Dimension: – Each cube has one or more dimensions, each based on one or
more dimension tables.– A dimension represents a category for analyzing business
data.– Typically, a dimension has a natural hierarchy so that lower
results can be “rolled up” into higher results
![Page 77: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/77.jpg)
Northern Region IT Professional Development Program 2010
Level, Fact Table, Measure & Schema
• Each type of summary that can be retrieved from a single dimension is called a level.
• Fact Table: contains the basic information that you wish to summarize.
• Every cube will contain one or more measures, each based on a column in a fact table that you’d like to analyze
– e.g. Unit Sales or Profit
• Schema: given that you use the dimension tables to group information from the fact table
![Page 78: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/78.jpg)
Northern Region IT Professional Development Program 2010
Creating a Data Cube
(1) File New Project…
(2) Select “Analysis Services Project”
![Page 79: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/79.jpg)
Northern Region IT Professional Development Program 2010
Creating a Data Cube
(3) From new solution, right click on Data Sources
(4) Create New Connection for AdventureWorksDW
(5) Select Default impersonation information to use the credentials you just suppliedfor the connection and click Next.
![Page 80: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/80.jpg)
Northern Region IT Professional Development Program 2010
Create Data Source View
(1) From new solution, right click on Data Sources View
(3) Assign Name “Finance” and click on Finish
(2) Select Data Source Select Tables
![Page 81: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/81.jpg)
Northern Region IT Professional Development Program 2010
Create Data Source View
(4) View Schema of new data source
Questions(1) What’s kind of schema?(2) What’s fact/dimension tables?
![Page 82: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/82.jpg)
Northern Region IT Professional Development Program 2010
Invoking the Cube Wizard
(2) Select option as shown in below pictures and click Next button
(1) Right-Click on Cubs New Cube…
(3) Select Data source and click Next
![Page 83: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/83.jpg)
Northern Region IT Professional Development Program 2010
Invoking the Cube Wizard
(4) Wait until Cube Processing finish, then click on Next button
(5) Identify Dimension & Fact
Select table “DimTime”
Select Fact & Dimension table (as shown in picture)
Click on Next button
![Page 84: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/84.jpg)
Northern Region IT Professional Development Program 2010
Invoking the Cube Wizard
(6) Select Time Periods
(7) Accept default measure
(8) Wait until Cube Wizard detected hierarchy
Click on Next button
![Page 85: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/85.jpg)
Northern Region IT Professional Development Program 2010
Invoking the Cube Wizard
(9) Accept default Dimension
(10) Assign Cube Name
Click on Finish button
![Page 86: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/86.jpg)
Northern Region IT Professional Development Program 2010
Deploying and Processing a Cube
(1) Build Deploy
(2) Waiting while processing
One of the tradeoffs of cubes is that SQL Server does not attempt to keep your OLAP cube data synchronized with the OLTP data that serves as its source. As you add, remove, and update rows in the underlying OLTP database, the cube will get out of date.
Re-Run Process
![Page 87: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/87.jpg)
Northern Region IT Professional Development Program 2010
Exploring a Data Cube
Click on Browser
• Drop a measure in the Totals/Detail area to see the aggregated data for that measure.• Drop a dimension or level in the Row Fields area to summarize by that level or dimension on rows. • Drop a dimension or level in the Column Fields area to summarize by that level or dimension on columns • Drop a dimension or level in the Filter Fields area to enable filtering by members of that dimension or level.• Use the controls at the top of the report area to select additional filtering expressions
Same as PivotTable in Excel
![Page 88: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/88.jpg)
Northern Region IT Professional Development Program 2010
Exploring a Data Cube
Drag into Total/Detail Area
Drag into Row Field Area
Define hierachy from Dim TimeCalendar Year-Calendar Quarter-Month Number of Year
Filter Area: Scenario Name
![Page 89: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/89.jpg)
Northern Region IT Professional Development Program 2010
SSAS: Exercise
• Create a data cube, based on the data in the AdventureWorksDW sample database, to answer the following question: what were the internet sales by country and product name for married customers only?
![Page 90: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/90.jpg)
Northern Region IT Professional Development Program 2010
References/External Links
(1) SSIS Tutorial: SQL Server 2005 Integration Services Tutorial http://www.accelebrate.com/sql_training/ssis_tutorial.htm
(2) SSAS Tutorial: SQL Server 2005 Analysis Services Tutorial http://www.accelebrate.com/sql_training/ssas_tutorial.htm
(3) MS SQL Server Data Transformation Services & Integration Services Chris Riley March 29, 2007
(4) Intro to SQL Server Integration Services & SQL Agent Jobs Stu Yarfitz Analyst/Programmer, Collaborative Data Services. FHCRC 6/18/2008
![Page 91: It ready dw_day3_rev00](https://reader036.vdocuments.mx/reader036/viewer/2022081518/546e79beaf795976298b570d/html5/thumbnails/91.jpg)
Northern Region IT Professional Development Program 2010
Thank you for your attention!