developing a cognitive assistant for audit plan...
Post on 22-Jun-2020
2 Views
Preview:
TRANSCRIPT
Agenda
Introduction
Research Problem
Proposed Solution - Framework
Experiment and Demo
Limitation and Future
2
Introduction
Mandated in two auditing standards: SAS No. 99 and SAS No. 109
Purpose: Discuss the susceptibility of the entity’s financial statements to material misstatement
due to fraud (Landis 2008, Hammersley et al. 2010, Beasley and Jenkins 2003)
Common procedure: Checklist in open-ended form
Importance: Generate more high-quality ideas on fraud risks as a team (Hammersley et al. 2010,
Carpenter 2007)
Affect auditors’ subsequent performance of audit
Current Issues:
Limitations with Checklist
Ineffective new ideas generation
Uncollected experience and expertise
Unreached additional external information
Audit Plan Brainstorming Session:
3
What we can do??
Audit Cognitive Assistant
Cognitive Assistant (Intelligent Personal Assistant)
Interact with users with speech-enabled technologies
Provide assistance by answering questions, making recommendations, and performing
actions (Canbek and Mutlu 2016, Hauswald et al. 2015, Myers et al. 2007, Garrido et al.
2010, Chen, 2015)
Commercial personal assistants (Mehrez, 2013): Apple Siri, Google Now, Microsoft’s
Cortana. Amazon’s Echo/Alexa IBM Watson
Cognitive Assistant (Intelligent Personal Assistant)
4
Advantages Of Cognitive Assistant In Audit Decision Plan Brainstorming:
Technically feasible
Advanced AI technologies
Big data support
Solution to a success of brainstorming meeting
Knowledge collection and organization (Knowledge Base)
Audit information retrieval and decision making support (Cognitive Computing)
(IBM 2016)
Recommendation on discussion procedure and topics (Adaptive Learning) (Myers
et al., 2007)
Audit applications / program connection (Invoking Apps) (Canbek and Mutlu
2016, Ebling 2016)
Quick interactions with auditors (Spoken Dialogue System )
Research Motivation5
QA
Proposed Method: Architecture of the Proposed Audit Cognitive Assistant
Automatic
Speech
Recognition
Query
Classifier
Question or
Action
text
Answer ShowAnswer
ExecuteAction
Action
Lucaindustry
Client
Position
Luca Luca
LucaRecommended Topics:General understanding, new events, business risks…
Processing…
You may also interested in:…
Query
Open an application
Inte
rface
Arc
hitectu
re
Modules:
• Automatic Speech
Recognition (ASR)
• Language Understanding
• Dialogue Management
• Natural Language
Generation
• Text-to-Speech synthesis
Audit Related Applications It Can Access
Web
SearchOpen (ACL,
IDEA…)Calculator
Open
standards
Open
templatesAudit
workpaper
Calendar …….backsta
ge
support
er
Knowledge Database
Knowledge
about users
Knowledge Base
DBMS
Unstructu
red data
Domain
Knowledge
6
Luca
industry
Client
Position
Luca Luca
Luca
Brainstorming
Discussion Topics
Recommended Topics:General understanding, new events, business risks…
Business risks
Processing…
You may also interested in:…
Query
Inte
rface
• General understanding
• New events/ areas
• Significant accounts
• Business Risks
• Financial statement
level risks
• Fraud Risks
• Going Concern
• Accounting policies
• IT controls
• Related Parties
• Internal Control
• Professional skepticism
• Materiality …
Risk Assessment Areas
General understanding
New events/ areas
Significant accounts
Business Risks
Fraud Risks
……
Recommended Risk
Assessment Areas
Based on Industry and Client
Examples of Recommended Sub-topics
General understanding
You may be interested in…
Company information
Management
Business strategy
Industry environment
Market competition
Economic factors
New areas/events
You may be interested in…
foreign operations
joint ventures
new agreements with
covenants
new acquisitions
new stores in other
countries (branches)
new general counsel
Cyber security
Significant Accounts
You may be interested
in…
Recom
mendation
Syste
m
Open an application
Recommender System of the ProposedCognitive Assistant7
Architecture of the Proposed Audit Cognitive Assistant
Lucaindustry
Client
Position
Luca Luca
LucaRecommended Topics:General understanding, new events, business risks…
Processing…
You may also interested in:…
Query
Open an application
Inte
rface
QAAutomatic
Speech
Recognition
Query
Classifier
Question or
Action
text
Answer ShowAnswer
ExecuteAction
Action
Arc
hitectu
re
Modules:
• Automatic Speech
Recognition (ASR)
• Language Understanding
• Dialogue Management
• Natural Language
Generation
• Text-to-Speech synthesis
Knowledge Base
DBMS
Audit Related Applications It Can Access
Web
SearchOpen (ACL,
IDEA…)Calculator
Open
standards
Open
templatesAudit
workpaper
Calendar …….backsta
ge
support
er
Knowledge Database
Knowledge
about users
Unstructu
red data
Domain
Knowledge
8
Question
Analysis
Query
Generation
Question
Normalized Question
Generated Query Answer
Matching
Answer
Extraction
Knowledge
base
Doc. With Candidates
Answer with highest score
Answer
Selection
Answer
NL Answer
Generating
Paired QA DBMS
Machine Learning Models
Answer
①
②
Question Answering System of theProposed Cognitive Assistant
① +②IR-based (Domain) QA
+ Knowledge-based
(Cognitive) QA
9
Keyword
extraction;
Punctuations,
abbreviations,
nouns and verbs
Question Answering System of theProposed Cognitive Assistant
Specialists
Materiality
Related parties
Going Concern
Client acceptanc
e
Things to be careful
Business understan
ding
Company informatio
n
history
management
executives
employee
compensation plan
events
Business
franchise/store
number
location
performance
business lines
main
new
products
competitor
market
strategy
strategy
old
new
risks
sales performan
ce
Economic factors
Weather
Season
IT security
Financial Reporting
Regulation/law
Controls
Political environme
nt
Proposed Questions and Answers Categories
Example: IT security
Q A
Do they have any security issue
before?
Customer‘s credit card
information leakage issue
What IT controls do they have? New head of IT, control over
POS systems in the stores;
control from store POS system
to servers at IT center
10
Experiment
Experiment Objective:
• Develop knowledge base for the proposed cognitive assistant
System supporting data:
• Recordings of four typical brainstorming discussions conducted in a big audit firm
• Additional cases can be integrated into the system
(Brian H. manager)
53And you mentioned that they're still trying to figure out the right strategy there
54the document also talks about a new strategy
55and transforming the business
56and I think they are trying to go from maybe a perception of a little bit more upscale,
57to more of the bar and grill.
(Brian F.Partner)
58It is interesting
59because when they started they were clearly a bar and grill restaurant
11
Question
Analysis
Query
Generation
Question
Normalized Question
Generated Query Answer
Matching
Answer
Extraction
Knowledge
base
Doc. With Candidates
Answer with highest score
Answer
Selection
Answer
NL Answer
Generating
Paired QA DBMS
Machine Learning Models
Answer
①
②
Question Answering System of theProposed Cognitive Assistant
① Domain QA:
PandoraBot②Cognitive QA:
IBM Watson
12
System Testing
Testing input: question asked by user
Testing output: generated answer
Using developed QA pairs to build Domain QA part:
Examples from Pandorabot (written with AIML):
Sample Q A
what is the number of stores and
franchise
700 stores: 650 company-owned franchise and 50
franchises
what are the main businesses
Main businesses include bar and grill, and Mexican
restaurants.
13
Limitation and Future Work
Limitation
Need large number of brainstorming meeting cases
Continue Work
Develop industry based recommendation system
Develop audit knowledge QA system
Design an “Audit Watson” for predictive audit risk assessment: case of
medical industry
14
top related