agile testing of data warehouse system in a non...
TRANSCRIPT
Agile Testing of Data Warehouse System in
a Non-agile Environment Swiss Testing Day 2015
Yury Chernov
10 March 2015
Classification: public
10 March 2015, public Page
Data Warehouse
2
Typical Requirements to DWH:
Make information easily acceptable – the contents must be
understandable
Present information consistently – the data must be credible
(data quality)
Adapt system to change – user needs, business conditions,
technology are all subject to change
Present information in a timely way
Secure and protect data
Business community should accept the system
DWH is a de-normalised, business-centric
database or set of databases in which the
data from different sources is collected and
stored in one uniform and consistent form
10 March 2015, public Page
DWH Characteristics
3
Attribute Database
OLTP (Online Transaction Processing)
DWH
OLAP (Online Analytical Processing)
Source of data
Operational data from original sources Consolidation data from various OLTP Databases
and files
Purpose of data Execution of fundamental business tasks Planning, problem solving, decision support
Type of data Snapshot of ongoing business processes Multi-dimensional views of business activities
Users Technical Staff Managers, Executives
Inserts & Updates Short and fast inserts/updates initiated by end
users/applications
Periodic long-running batch jobs refresh the data
Queries Standardized, simple and short queries Complex queries involving aggregations
Processing Speed Fast Batch jobs and complex queries that may take
hours
Emphasis Insert, update, delete Select-Retrieval
Space Requirements Relatively small Large due to aggregated structures and history data
Database Design Highly normalized Typically de-normalized
10 March 2015, public Page
DWH in Swiss Exchange Landscape:
Two Roles of DWH as a Testee
4
Role 1: A complicated component for business end-users
Role 2: An important downstream point in E2E testing
SWX DWH is not a database of corporate information
(like SAP), but a storage of trading and reference
data and a set of applications (Billing, Spread
calculation etc.)
10 March 2015, public Page
DWH Architecture
5
Source Stage Store Mart Front-end
Data sources ETL layer (extract, transform, load) Complete data User access layer User access tools
Analytical Data Process Data Operational Data
10 March 2015, public Page
DWH Testing
6
Test objects
Data transformations
Data integration
Data quality
Performance & scalability
User-acceptance
Tools
Quality Center
PCMS
Test Automation Framework
Data profiling scripts
PowerCenter
«Testers are people who realize that things
can be different»
Jerry Weinberg,
“the grandfather of Agile Programming”
10 March 2015, public Page
Agile Testing
7
1. Customer Satisfaction
2. Flexibility
3. Short Iterations
4. Daily Teamwork
5. Motivation
6. Face to Face Communication
7. Working software
8. Sustainability
9. Technical Excellence
10. Simplicity
11. Self-organisation
12. Regular Self-reflexion
Agile Practices
Agile Principles
Agile Values
1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan
An agile tester: “a professional tester who
embraces change, collaborates well with
both technical and business people, and
understands the concept of using test to
document requirements and to drive
development.”
Lisa Crispin & Janet Gregory in “Agile Testing”
10 March 2015, public Page
DWH Agile Testing
8
Swiss Exchange Landscape
Other Components DWH
Detailed functional specifications No clear boundaries between Verification
(are we building the product right?) and
Validation (are we building the right
product?) – requirements are often not
formalised
Testing includes component, integration,
system & E2E phases, all in special separate
environments
DWH is being tested directly in full-blown
E2E environment
Components are developed across system
projects
DWH is part of system projects and has
additionally ad-hoc deliveries for end-users
10 March 2015, public Page
Practical Lessons Learned
9
Lesson 1: Note everything.
Lesson 2: Maintain your testware permanently.
Lesson 3: Plan enough effort to maintain your test environments.
Lesson 4: Automate as much as possible.
Lesson 5: Your test data should be as representative as possible.
Lesson 6: Your bug tracking system is the best communication mean.
Lesson 7: Pay special attention to the data quality checks.
10 March 2015, public Page
Lesson 1: Note everything
10
Best of all is to note information directly in the test specification.
Your test specification becomes often the most reliable document
about the real system functionality.
Everything should be noted
An agile tester: “a professional tester who embraces
change, collaborates well with both technical and
business people, and understands the concept of
using test to document requirements and to drive
development.”
Lisa Crispin & Janet Gregory in “Agile Testing”
10 March 2015, public Page
Lesson 2: Testware
11
For the agile testing the testware should be always operational ready.
For a project manager the testware upgrade is the lowest priority task in
his project, but you will use this testware in next projects.
Test case should describe what to test, rather than how to test.
Keep the logic of tesware reasonably simple and thus maintainable.
Maintain your testware permanently
10 March 2015, public Page
Lessons 3: Test Environments
12
Test environment should be controlled.
Test environments should be always available.
A running test environment makes 50% of the job.
Plan enough effort to maintain
your test environments
10 March 2015, public Page
Lesson 4: Test Automation
13
With agile testing you will run your scripts much more often, than you
are thinking.
Pay more attention to the data profiling scripts.
Automate as much as possible
SWX:
1. Test Automation Framework for generation of the
trading data.
2. SQL Scripts to compare the content of different DBs.
3. Daily data profiling tasks
10 March 2015, public Page
Lesson 5: Test Data
14
Use a combination of production and synthetic data.
Every problem you saw with your data will once appear in production.
Developers will tend to accuse your test data in all the problems you
see, however in most of the cases they will turn out to be bugs.
Your test data should be as
representative as possible
10 March 2015, public Page
Lesson 6: Bug Tracking System
15
Note every problem in your bug tracking system:
• your will not forget the details
• developers will take it more serious (they don’t like registered problems)
• it can be canceled, if not relevant.
Force (if possible ) developers and business users to register problems in the
bug tracking system.
Avoid using multiple Excel-sheets to register problems.
Your bug tracking system is the
best communication mean
10 March 2015, public Page
Lesson 7: Data Quality
16
Automate data quality checks
Establish data checks scripts that run daily.
Check data quality of the source data files.
Pay special attention to the data
quality checks
"Program testing can be a very effective
way to show the presence of bugs, but is
hopelessly inadequate for showing their
absence."
Dr. E.W.Dijkstra
10 March 2015, public Page
Summary
17
1. Swiss Exchange is mostly being developed and tested with the classical
Waterfall model.
2. DWH has many specifics that lead to agile testing.
3. To properly answer this challenge testers should have a reasonable
common sense based practical approach.
4. Positive and negative experience -> “lessons learned/good practises”
are helping us to successfully test DWH and avoid production problems.
Thank you for your attention.