![Page 1: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/1.jpg)
Carolyn M. Engstrom
![Page 2: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/2.jpg)
Gain a new perspective on the problem of IT Data Analytics
Leave with inspiration and information about how to apply data analytics to achieve value
![Page 3: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/3.jpg)
![Page 4: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/4.jpg)
The cobbler is the IT department which uses his skills and tools to
make shoes.
Shoes are metrics, output, analysis, etc.
Shoeless children are internal processes.
IT doesn’t apply tools and skills to
meet it’s own goals
Audit and Compliance are child protective
services. “Your children have no shoes!!”The broader
organization helps design them and
uses them.
![Page 5: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/5.jpg)
Big Data centric Metrics focused Necessary evil of compliance Effectiveness dominates Efficiency lags Structured, centralized data Enterprise solutions Security Event and Incident Management Analytics are afterthoughts of implementation
![Page 6: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/6.jpg)
Data quality worries Data efficiency worries Need to predict, forecast Historical reporting Siloed knowledge of business process “Gartner Says Power Shift in Business
Intelligence and Analytics Will Fuel Disruption”
CIO: 21 Data and analytics trends that will dominate 2016
![Page 7: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/7.jpg)
• Statistical• Predictive Models
• Really big data!• Lots of sources!• Really important
issues to solve!
• End-user focused• Reporting• Summarize• Drill Down
• Outside Data Sources
• Unstructured Data• Extract, Transform,
Load
Source: “What Kind of Big Data Problem Do You Have?” SAS, 2014
Reac
tive
Proa
ctiv
e
Large Big Data
Dat
a Ca
pabi
lity
Data Size
Big Analytics Big Data Analytics
Business Intelligence
Big Data Business
Intelligence
![Page 8: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/8.jpg)
They come in many sizes Big: $$$◦ Aggregated from External Sources◦ Primarily Big Data Business Intelligence
Medium: $$-$$$◦ Aggregated from Internal and External Sources◦ Operational and Security Information
Small: $◦ Internal Accumulation◦ Risk Assessment◦ Context, Calibration, Criticality
“Actionable Security Intelligence From Big, Midsize and Small Data “ by C. Warren Axelrod, Ph.D., CISM, CISSP – ISACA Journal, 2016
![Page 9: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/9.jpg)
Achieve Insight Uncover Meaning Improve Assurance/Effectiveness Improve Efficiency Identify Trends Demonstrate Progress Prototype Requirements Improve Data Integrity Unlock Knowledge Management
“A Practical Approach to Data Analytics”, ISACA, 2011
![Page 10: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/10.jpg)
Black box auditing◦ Frameworks◦ Methodology◦ Audit procedures◦ Standards of fieldwork
Evolving data analytics skillset Reports lack a persuasive story, meaning or
context Unique exposure to data, processes, and
risks
![Page 11: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/11.jpg)
Source: CEB Audit Leadership: Peer Feedback- Data Analytics Vendors 2014
![Page 12: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/12.jpg)
Define a population◦ Controls or risks
Information Provided by Client (IPE)◦ Population integrity
Non-statistical Sampling: based on frequency◦ Annual, Semi-annual, Quarterly, Automated = 1◦ Monthly = 2◦ Weekly = 5◦ Daily = 15◦ Many times daily = 25
![Page 13: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/13.jpg)
Statistical Sampling◦ Confidence Intervals, 90% or 95%◦ Mathematical function identifies sample size◦ Not frequently used
100% population analysis◦ 1 source- IPE◦ 2 sources or more- Data Integrity◦ Removes population bias◦ Provides quantifiable measure of effectiveness
Assessment of exceptions All of these techniques support an auditor’s
conclusion
![Page 14: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/14.jpg)
Source: CEB Audit Leadership: Peer Feedback- Data Analytics Vendors 2014
![Page 15: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/15.jpg)
![Page 16: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/16.jpg)
Expand your perspective on data◦ Transaction◦ Trending◦ Continuous Monitoring
If data is valuable, for the love of goodness, DO NOT USE A WORD DOCUMENT as a source of truth… EVER.
![Page 17: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/17.jpg)
![Page 18: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/18.jpg)
Control: For SOX in-scope apps job completion is monitored and abends are recorded in ticket software and resolved.
Batch process extracts job
fails from log
Employee selects a sample of
25
Employee searches for ticket
Employee records results in a Word doc. Embeds job
log object(s)
Monthly Quarterly 25 manual times 4Q x n apps
Audit used logs to
select their own
sample
![Page 19: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/19.jpg)
Testing covers only failures◦ How many jobs ran successfully?
Only applied to SOX applications Manual process◦ Required about 2-3 hours per quarter per app◦ Multiple control owners
Audit coverage was minimal % of population Files maintained all over the network
![Page 20: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/20.jpg)
Batch process extracts job completions
and fails from log
Monthly
Use data prep software to format logs
User extracts tickets
Compares
Exceptions: not timely, no ticket
Sends Exceptions
Report sent to control owner
![Page 21: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/21.jpg)
Redesign cost about $2000 for data prep Time investment of about 40 hours Quantitative assurance◦ 100% SOX population coverage◦ 100% exception coverage
Context of success, failures, exceptions (%) Correct data quality issues Centralize file storage Increase frequency to monthly from quarterly
but decrease time
![Page 22: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/22.jpg)
Build a table for jobs and attributes◦ Interfaces Data flow of confidential data Data flow of financial data◦ Report Integrity◦ Job number◦ Criticality
Build knowledge management Use data visualization rather than reports
![Page 23: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/23.jpg)
![Page 24: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/24.jpg)
Narratives about 6-20 pages long Topics◦ Access Controls◦ Change Management◦ Interfaces ◦ Job Resolution◦ Infrastructure Identification (asked to update xls) App servers Database servers and instances Servers (OS, location)
Identified Business Processes, but not financial statement accounts or disclosures
![Page 25: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/25.jpg)
Narrative of an actual process Identify financial statement accounts and
disclosures Identify key controls May identify key reports by name Identify information on interfaces
![Page 26: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/26.jpg)
Productionize Application Narrative◦ Change management application attributes◦ Created report out of the application◦ Improved population for change management
controls Foster Audit Knowledge Management ◦ Key Reports◦ Interface information to Chart of Accounts◦ Financial Statement Line Items◦ Custom Report for Review
![Page 27: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/27.jpg)
Create relationships among data that was previously locked
Transform unstructured data Enforce consistency Content is more accessible Less data to maintain Improve efficiency and effectiveness of
existing tools
![Page 28: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/28.jpg)
![Page 29: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/29.jpg)
No previous defined vulnerability management process
Select a large-scale tool for vulnerability identification
Delays in projects due to incomplete network topography
![Page 30: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/30.jpg)
Use Nessus to scan sample of servers (20) Collect data to baseline scores Use scripts to collect ◦ Patch levels from servers ◦ Event log entries◦ Registry settings◦ Customized reporting
Use data to clarify business requirements◦ Roles◦ Communication requirements◦ Documentation
![Page 31: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/31.jpg)
More quantifiable data than initial business case
Established expected baselines Resourcing and timelines Calculated revised Return on Investment Defined a process Verified business requirements
![Page 32: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/32.jpg)
![Page 33: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/33.jpg)
1. Map regulatory/oversight requirements to internal controls
2. Inventory and leverage existing data sources 3. Use existing, free, or low cost tools4. Analyze Baseline◦ Data Flow◦ Data Integrity◦ Return on Investment
5. Re-baseline and productionize (governance)◦ Automation◦ Workflow
![Page 34: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/34.jpg)
Don’t overlook unstructured data Unlock your small data◦ Gather and update effectively◦ Focus on context and criticality
Audit can be great sources of small data, but know the audit approach
Leverage the same data sources for different risks and insights
![Page 35: Carolyn Engstrom - IT Data Analytics: Why the Cobbler's Children Have No Shoes](https://reader031.vdocuments.mx/reader031/viewer/2022021813/587ce1011a28abff0b8b6ccd/html5/thumbnails/35.jpg)
Data-Driven Security: Analysis, Visualization and Dashboards by Jay Jacobs and Bob Rudis (book)
Threat Modeling: Designing for security by Adam Shostack (book)
Database Debunkings Fabian Pascal (blog) Dresner Advisory Services 2016 End User
Preparation Market Study (Market Research) Storytelling with Data by Cole Nussbaumer
Knaflic (book and blog)