william j. papanikolas, cisa, cfsa western michigan ... · pdf file1 how to audit the system...
TRANSCRIPT
![Page 1: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/1.jpg)
1
How to Audit the System Development Life Cycle
William J. Papanikolas, CISA, CFSA
Western Michigan Chapter ISACA
October 20, 2011
![Page 2: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/2.jpg)
2
Today’s Agenda
Defining SDLC
How to Impact the SDLC Process
What to Audit at Each Step
![Page 3: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/3.jpg)
3
Defining SDLC
The System Development Life Cycle (SDLC) is the entire systems process from identifying a need through the final implementation of a solution.
SDLC is one of the best places for an auditor - and yet one of the least audited.
![Page 4: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/4.jpg)
4
Defining SDLC
Successful SDLC projects are measured three ways:
Creating a Quality Product
Completing at Budgeted Cost
Completing on Approved Timetable
NOTE: The majority of SDLC projects fail to achieve even two of these goals!
![Page 5: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/5.jpg)
5
Defining SDLC
Project
Initiation
Business
Requirements
Definition
Technical
Requirements
Definition
Software
Selection /
Coding
Testing Data
Conversion
Training and
Documentation
Final
Implementation
![Page 6: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/6.jpg)
6
How to Impact the SDLC Process
Creating a Quality Product Provide assurance the final product is going to
deliver what has been promised
Ensure proper controls (automated and manual) have been designed into the new process
Completing at Budgeted Cost Ensure SDLC oversight includes controls for costs
Completing on Approved Timetable Ensure SDLC oversight includes controls for
timeliness
![Page 7: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/7.jpg)
7
Project Initiation
Project champion determined.
Project charter developed.
High level timelines and budgets determined.
Project team assigned; roles and responsibilities established.
Project monitoring and accounting set up.
![Page 8: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/8.jpg)
8
Project Initiation - Audit Ideas
Quality Product
Appropriate stakeholders are involved.
Project champion represents the key stakeholders.
Project is consistent with the organization’s strategic plans.
![Page 9: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/9.jpg)
9
Project Initiation - Audit Ideas
On Time and On Budget
Budget was properly determined (watch out for approval cutoffs!).
Timeline is realistic given project magnitude and past organizational experience.
Appropriate metrics and reporting schemes are developed.
![Page 10: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/10.jpg)
10
Business Requirements Definition
Primary system functions defined.
Usability targets established (e.g., 24x7, 1 second response time).
Management reporting requirements understood.
Regulatory and legal implications considered.
End user screen requirements determined.
![Page 11: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/11.jpg)
11
Business Requirements Definition - Audit Ideas
Quality Product
Appropriate stakeholders are represented.
Security requirements are defined.
Automated and manual controls are considered.
![Page 12: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/12.jpg)
12
Business Requirements Definition - Audit Ideas
On Time and On Budget
Project plan and budget remain realistic given business requirements.
Business requirements do not overly rely on new and/or unproven technologies (e.g., a requirement that all transactions will process over the intranet).
![Page 13: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/13.jpg)
13
Technical Requirements Definition
Processing platform(s) determined
Necessary hardware acquisitions outlined.
System capacity requirements understood (both processing speed and data storage).
Network modifications defined.
Data structures created.
![Page 14: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/14.jpg)
14
Technical Requirements Definition - Audit Ideas
Quality Product
Technical requirements support the business requirements.
Members of all impacted technical units represented.
Technology assumptions are properly validated through internal experience or external site visits.
Links to existing applications are defined and controlled (e.g., control totals)
![Page 15: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/15.jpg)
15
Technical Requirements Definition - Audit Ideas
On Time and On Budget
Project plan and budget remain realistic given technical requirements.
Lead times for purchasing, receiving, installing and testing new hardware have been properly reflected in the timeline.
![Page 16: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/16.jpg)
16
Software Selection/Coding
Request for Proposal created.
Vendor and software selection criteria established.
Contract terms established.
Programming teams assigned for coding and modification.
Software loaded in test environment.
![Page 17: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/17.jpg)
17
Software Selection/Coding – Audit Ideas
Quality Product RFP and vendor assessments come directly from business and technical requirements.
Selected vendor has experience in your industry, with companies your size, and with similar setups.
Vendor is financially stable and will be around for long term support (alternatively, the source code could be owned by your organization).
Proper change management and security controls are set up for the coding environment.
![Page 18: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/18.jpg)
18
Software Selection/Coding – Audit Ideas
On Time and On Budget
Vendor contract terms are favorable, and include clauses on cost overruns.
Vendor contract includes rewards/penalties for project timeliness.
Project plan appropriately reflects the resources and time necessary to install, code and modify.
![Page 19: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/19.jpg)
19
Testing
Unit testing completed for each system element.
Integrated testing completed for each system module.
System testing completed for overall system and related interfaces.
Stress testing completed for online performance and data storage/retrieval.
End user testing completed.
![Page 20: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/20.jpg)
20
Testing - Audit Ideas
Quality Product
All testing is performed in an appropriate environment with adequate security.
All issues noted during testing are communicated to the proper owner within the project.
Test cases reasonably reflect the environment as it will appear in production.
Change management controls are in place as system elements progress through the testing cycle.
![Page 21: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/21.jpg)
21
Testing - Audit Ideas
On Time and On Budget
Resolution of test issues is focused on items that are necessary to achieve business or technical requirements (not all issues must be solved prior to going live!).
Project plans are properly updated to reflect issues noted in testing that must be resolved.
![Page 22: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/22.jpg)
22
Data Conversion
Data from the old system(s) is properly cleansed prior to conversion.
Converted data is evaluated to ensure it is accurate and complete.
![Page 23: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/23.jpg)
23
Data Conversion - Audit Ideas
Quality Product
Data is accurately mapped from the old system to the new.
Key data elements are screened using software (or manually in some cases) to ensure anomalies are removed.
After conversion, sample data reflects accurate transfer.
Control totals of key data fields/tables show consistency in the old and new data structure.
![Page 24: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/24.jpg)
24
Data Conversion - Audit Ideas
On Time and On Budget
Project plans are properly updated to reflect issues noted in data conversion that must be resolved.
![Page 25: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/25.jpg)
25
Training and Documentation
Users, operators, database administrators, management, etc. receive the training required to operate and use the system.
Documentation is provided for all users and operators of the system.
![Page 26: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/26.jpg)
26
Training and Documentation – Audit Ideas
Quality Product
Training addresses both system usage and business process.
Training includes all affected parties.
Training is provided close enough to implementation to allow participants best retention.
Documentation (online and paper) is organized in a way to be useful to users and operators.
![Page 27: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/27.jpg)
27
Training and Documentation – Audit Ideas
On Time and On Budget
Training and documentation are properly included in the project plan and budget.
![Page 28: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/28.jpg)
28
Final Implementation
Final system running in the production environment.
New hardware, networking, etc. comes online.
Business processes change over to accommodate new system.
![Page 29: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/29.jpg)
29
Final Implementation - Audit Ideas
Quality Product
Promotion to production environment follows established change management procedures.
Parallel processing with old system(s) commences.
Help desk and “swat teams” are in place.
System backout procedures are established.
![Page 30: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/30.jpg)
30
Final Implementation - Audit Ideas
On Time and On Budget
Final costs are captured and summarized (watch out for implementation problems being defined as “on-going maintenance”).
Project teams are closed down as the implementation continues.
![Page 31: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/31.jpg)
31
What’s Next?
Post-Implementation Review
Lessons Learned
Final Reporting
![Page 32: William J. Papanikolas, CISA, CFSA Western Michigan ... · PDF file1 How to Audit the System Development Life Cycle William J. Papanikolas, CISA, CFSA Western Michigan Chapter ISACA](https://reader033.vdocuments.mx/reader033/viewer/2022051720/5a7869a67f8b9a63638eb620/html5/thumbnails/32.jpg)
Questions?