making data science pay: mastering the challenges of ...€¦ · 13/06/2018 · making data...

Making data science pay:

Mastering the challenges of analytics operations

Michel DebicheGuest Analyst, STAC

[email protected]

Copyright © 2018 Securities Technology Analysis Center LLC

®

Cognitive Reset Part 1: Window management technology


®

Cognitive Reset Part 2: Window management context


®

Investment Process

Gather

information

Digest

information

Make

decisionsExecute

decisions

Actions

Results

Info


®

Pressures

• Scale

• Volume

• Variety

• Density

• Computational complexity

• Velocity of innovation

• Cost

• Regulation


®

Dimensions of scale

• Scale• Volume

• Variety

• Kinds of data: structured, unstructured, text, binary

• Data entities: Millions of time series

• Density

• Transactions in microseconds

• Simultaneous transactions on multiple channels

• Computational complexity

• NLP, Image processing, AI

• Velocity of innovation

• Competitive pressures: New datasets, new models, new technologies

• Evolving opportunities

• Feedback loops


®

Responses

• DevOps

• Data Lake

• Open Source

• Big Data

• Data Science

• AI


®

Issues

• Model Factories: Hundreds of models with nowhere to go

• Redundant engineering

• Open source interoperation and upgrade nightmares

• Murky, expensive data lakes contributing little value

• Skills mismatches

• User resistance to new technologies

• Data lineage, audit trails


®

Goals

• Maximize returns

• Minimize risk

• Market risk

• Model risk

• Systems risk

• Data risk

• Operational risk (people)

• Maximize productivity


®

Principles

• Optimize use of resources

• People

• Time

• Data

• Technology

• End-to-end process design

• Agility

• Constant improvement


®

Industrial Engineering

• Similar challenges and goals

• Eventually came to software engineering as DevOps

• Need to carry paradigm over to full data-to-decision pipeline

• Why is it so hard?


®

DevOps: Elegant Concept


®

DevOps: More complicated to implement


®

So let’s think about QuantOps™


®

Investment Process

Gather

information

Digest

information

Make

decisionsExecute

decisions

Actions

Results

Info


®

Investment Process, Expanded

Research data, develop and test models

Devops for data prep, analytical

functions, API

Production pipeline: data to

curated feature

Model scoring engine

Model testing manager

Data

Core

Research Data

Feature

updates

Test backlog

Data

Model suite

updates

Features

Features

Scores

Results

Ad hoc data

ingestion

Ideas

Function

library

Feature

definition

Model

definition

Model

Repository

Feature

preparation

code

Function

definition

Results

Results

Results


®

1

7

A Unifying Paradigm: QuantOps as a DAG


®

A Unifying Paradigm: QuantOps as a DAG

• Standardize the connections

• Carefully define the data APIs

• Then all the technology is pluggable

• Makes it possible to efficiently address:

• Orchestration

• Data lineage

• Monitoring

• Audit trails

• Automated code generation and testing


®

Where does STAC fit in?

• Implementing analytics ops is a big commitment with big payoffs

• Biggest challenge: effective communication, change management

• Design needs to be process-oriented and based on user needs

• Technology needs to respond to process requirements, not vice versa

• Emerging STAC roles:

• Facilitate dialogue & training on analytics ops challenges & best practices

• Accelerate technology selection based on community-source standards

driven by process-oriented model of the investment process

• Let us know if you want to be involved!

making data science pay: mastering the challenges of ...€¦ · 13/06/2018 · making data...

Documents