sentinel week 5 h4d stanford 2016
TRANSCRIPT
Team Sentinel
● Team members:○ Jared Dunnmon○ Darren Hau (Chief Animator)○ Atsu Kobashi○ Rachel Moore
● Cumulative # of interviews: 51 + 11○ Users: 5 Experts: 6
● What we do: Enable rapid, well-informed decisions by establishing a common maritime picture from heterogeneous data
○ Open and automated data aggregation (i.e. incorporate open source data)○ Flexible layering and filtering with improved UI/UX○ Enhanced intel through contextualization and easily accessible, common database○ Identifying deviations from baseline○ Utilizing historical data
● Why it matters:○ Information overload○ A2/AD prevents deployment of traditional ISR in a timely manner○ Data aggregation platforms and database access in PACOM appear extremely manual
● Military Liaisons○ John Chu (Colonel, US Army)○ Todd Cimicata (Commander, US Navy)
● Problem Sponsor○ Jason Knudson (Lieutenant, US Navy
7th Fleet)● Tech Mentors include:
○ Elston ToChip (Palantir)
QUOTE OF THE WEEK
“Navy Acquisition: Using yesterday’s technology... tomorrow”
QUOTE OF THE WEEK
● Customer Workflow
● Customer Discovery
● Procurement, Deployment
● Mission Model Canvas + Value Props
● Technology Environment
● MVP
N2
Analysis
Strategic Decisions
CUB
Task Forces
Data Acquisition (among other things)N3
Operational Decisions
Information aggregation + analysis platform
Customer Discovery - JOC SWO Workflow
Main screens
Customer Discovery - JOC SWO Workflow
Hmm...we need the USS Stockdale. PacFleet report says it should be in transit.
Customer Discovery - JOC SWO Workflow
GCCS-Mconsole
Customer Discovery - JOC SWO Workflow
Dang...maintenance last night and I can’t log in now.
Customer Discovery - JOC SWO Workflow
Customer Discovery - JOC SWO Workflow
Customer Discovery - JOC SWO Workflow
Hey Max, the GCCS console is logged out. Can you start it up again?
PACOM civilian
Customer Discovery - JOC SWO Workflow
Sure!
Customer Discovery - JOC SWO Workflow
Customer Discovery - JOC SWO Workflow
Customer Discovery - JOC SWO Workflow
There you go!
Customer Discovery - JOC SWO Workflow
Hey Max, why is the ship still in port? This info isn’t up-to-date.
Customer Discovery - JOC SWO Workflow
Can you ask them to update this?
Customer Discovery - JOC SWO Workflow
Yeah, hold on...
Customer Discovery - JOC SWO Workflow
Customer Discovery - JOC SWO Workflow
PacFleet unit manager
Hey Lauren, can you tell them to update this ship’s location?
Customer Discovery - JOC SWO Workflow
7th Fleet
Hey Phil, can you get the new position for these guys?
Customer Discovery - JOC SWO Workflow
Sure!
Customer Discovery - JOC SWO Workflow
*Brrring*
Down the chain of command...
Customer Discovery - JOC SWO Workflow
Okay, the OS put in a new latitude and longitude.
Customer Discovery - JOC SWO Workflow
Okay, it’s done!
Ah, there it is.
Hypotheses Experiments Results Action
There are no programs in the works to effectively tackle problems
REFINED
- Interviews with PMW 150 APM, Gary Robinson (Scitor), Raymond Britt (BAH), Chuck Wolf (DHS)- Site visit to Alameda D11 Coast Guard command center
- Discovered Quellfire, an object-oriented database being rolled out- Other programs such as SeaVision, DARPA Insight- Existing tools are clunky and slow, with many holes in data due to need for manual input
- Get unclassified version of tool interfaces- Focus on building applications on top of infrastructure to enable rapid, inexpensive updates
Lack of local data storage + bandwidth is a problem
VALIDATED
- Interviews with CDR James Dudley (N2), OSCS Roundtree, Gary Robinson (Scitor)- Engagement with Jason Knudson (N2)
- GCCS limited to 250 MB hard drive -> only store data for about a week- GCCS connectivity drops in/out frequently
- Develop separate MVP to test local storage
Customer Discovery
Procurement Process...see next slides
Core Navy Procurement Process
PACOMTo win a war, we need to have awareness of potential adversary's disposition of forces within the area we intend to operate and be able to maintain that through all phases of the conflict (Joint Intelligence Preparation of the Environment)
PACFLT Use the Navy in 3rd and 7th Fleet to conduct JIPOE
7th Fleet Direct ships, aircraft, submarines, marines, and other sensors to conduct JIPOE
7th Fleet N2
Task, Collect, Process, Exploit, and Disseminate and maintain JIPOE for C7F
7th Fleet N2, LT
Knudson
Identify potential operational gaps and determine possible ways to fill those gaps
1. Operational Requirements flow down from PACOM and is interpreted at each level:
Operational Requirements
USFF
PACFLT
7th Fleet Do I have the tools to accomplish my Operational Requirement?
Yes No
YAY, Done
Does PACFLT have the money and/or resources to fund it?
Send Acquisitions Requirement to PACFLT
Yes No
YAY. Validated and resourced. Done.
PACFLT “endorses” requirement, sends to US Fleet Forces Command
Is USFF able to fund or resource this requirement?
Yes No
YAY. Validated and resourced. Done.
Send to OPNAV
OPNAVIs there an existing Program of Record?
No
YAY. DoneMake new POR and include in Navy’s budget via
SECNAV, SECDEF.. Send to Congress.
Congress Budget approved?
Yes
Acquisition Requirements
Congress Budget approved?
Yes
OPNAV
PMO
USFF
Force Commands
PACFLT
7th FLEET
Money flows from SECDEF to SECNAV to CNO/OPNAV
Primes/ NAC
Program Management Office decides who to tap for production/development
A government contractor (Boeing, Lockheed, etc.) or Naval Acquisition Command (SPAWAR, NAVSEA, etc.) builds this system
Product made available to US Fleet Forces Command to issue to Navy units
SURFFOR, SUBFOR, and IFOR man, train, and equip using 2-year money
No GG
PACFLT receives resources from the appropriate force command
7th FLEET GETS SOMETHING!!!! …. Many YEARS later…. YAY!!!!
Program Execution
Key Acquisition Paths
● Several potential deployment strategies○ Linking in with an existing POR (PMW-150, etc.)
■ Pros: Allocated funding, long-term integration plans■ Cons: Long timescale, getting in the door■ POCs: ONI, SPAWAR (Stan Kowalski), Primes■ Source of Excitement: TBD
○ Rapid Acquisition Pathways (Limited Objective Experiments, Rapid Reaction Technology Office)■ Pros: Speed, Close to user, Don’t have to go through Navy (other services
work)■ Cons: Limited spending authority■ POCs: 7th Fleet (Jason Knudson), DHS (Chuck Wolf)■ Source of Excitement: Rapid deployment, changed acquisition model
○ DARPA ■ Pros: Development mindset, existing programs (Insight) that are well-aligned,
deployment authority/capability to pay for deployment to end-users■ Cons: stepping on toes, limited number of PMs■ POCs: Craig Lawrence (ADAPT)■ Source of Excitement: Directly solving a problem as opposed to many-year
process
Sample Deployment Path (Software, POR Path)
1. Operational testing to make sure meets military specs (engage SPAWAR for this)a. Ensure NSA-standard Information Assurance (IA)
i. Lock down system and codeii. Make sure no category 1,2,3 in code - backdoors, exceptions, etc.
b. Observe appropriate NIST protocols (TBD)2. First, limited deployment to evaluate functionality (on testbed system or specific asset)3. Then, if integrated into a POR:
a. Deployed on whatever platform is neededb. Moves into sustainment phasec. Think about disposal & replacement--we want continuous improvement!
4. IT installs where requireda. Technical support install software and make sure up and runningb. Maintains communications systems and networks
5. Personnel training for system operation and maintenancea. CTMs focus on maintaining classified systems & special collections abilities
WE WILL BE GETTING MORE DETAIL ON THIS GOING FORWARD!
Research- Interviews to assess needs, organizational dynamics, procurement strategy- Site visits to see current practices-Understanding current workflow
Prototype- Integrate sensor feeds of interest into prototype platform- Compile existing data resources- Create representative “fake” datasets- Evaluate relevant ML algorithms for prediction and rules for push alerts- Iterate on human-machine interaction
Strategic Decision MakersVADM Joseph AucoinADM Scott Swift (PacFleet)ADM Harry Harris (PACOM)
Analysts (N/J2)E.g. Jason Knudson, John Chu, Jed Raskie, Joseph Baba Operators (N/J3)CDR Chris Adams (7th Fleet)
Planners (N/J5)Jose Lepesuastegui (N25)
- Common and consistent view of the Area of Responsibility (AOR)
- Timely operational decisions
- Decreased time to predict hot spots, ID & differentiate threats
- Reduced time for analysts to find information and draw conclusions
- Prototype operability + demonstrated scalability
Data Fusion/Sensor Integration Software (THIS SECTION IS A WORK IN PROGRESS!)
- Build solution that integrates with current systems (e.g. GCCS, QUELLFIRE, FOBM)
- Work with PMs and key influencers to determine optimal funding/dissemination avenues
- Deploy prototype, confirm buy-in and update features
- Scale deployment, improve product as necessary
Fixed- Buying proprietary data- Software tools- Evaluation of commercial products
Prototyping- Existing sensor platforms and feeds- Academic research- Existing data fusion platforms
Scaling- Available commercial + military data- Existing database tools (Palantir, AWS)
- Need commanding officer to confirm decision-making benefits
- Need intelligence officers from ONI / N2 and operators from N3 to confirm effectiveness of insights
- Need IT approvals to integrate into systems
- Need support of commercial partners if want to leverage their platforms
-Need support of existing PMOs to make sure we’re not duplicating work
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In
Deployment
Value Proposition
Key Activities
Key Resources
Key PartnersMilitary- 7th Fleet + designated sponsor- NPS/ONR- Acquisition Personnel- Existing PORs (Insight, PMW-150, Quellfire, SeaVision, FOBM)
Commercial- Distributed sensor platform companies (i.e. Saildrone, AMS)- Data analytics (i.e. Palantir, Google)
Academic- Universities (i.e. University of Hawaii)- National Labs (Lincoln Labs, Sandia)
Other- IUU fishing + anti-smuggling stakeholders (i.e. Coast Guard, PNA)- Disaster relief agencies
Mission: Enabling Rapid, Well-Informed Decisions from Heterogeneous Data
Testing- 7th Fleet assets for pilot- Research barge- Access to model analyst data interface- Access to sample incoming sensor feeds
Variable- Travel for site visits, pilots- R&D personnel-Development
IMPROVE TACTICAL AND STRATEGIC DECISION
MAKING VIA BETTER DATA HANDLING
(1) Rapid Strategic Decisionmaking via Improved Reporting and Coordination
(2) Improved Tactical Decision Making via Timely, Accurate Information Sharing
(3) More Effective Analysis via Searchable, Visualizable Data Integration (Layering & Filtering)
(4) Predictive Intel and Alerts (e.g. Machine Learning)
ENHANCE INCOMING DATA STREAMS
(1) Improved Collection of Existing Data Streams (e.g. Fishing Broadcasts)
(2) Painless Incorporation of Multiple New Sensing Modalities
(3 Integration of Incoming Data Streams with Existing Object-Oriented Database and Historical Datasets
Products& Services- Timely data- Good UI/UX
for presenting data
- Streamlined reporting process
- Improved coordination across ranks
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- Good UI/UX- Platform
incorporates more data streams
- Platform is robust and can handle drop out of data streams
- Allocate assets- Identify,
eliminate threats
- Predict hot spots
- Safety of team- Projecting
peace, stability in region
- More informed decisions
- Faster decisions- Decisions made
on most up-to-date info
- Poor quality/lack of data
- Latency of data -> insight
Admiral/Strategic Decision Maker
Value Proposition Canvas
Customer persona:
● 3 or 4 star admiral● Born in late 1950’s● Have their own office on-base● Gives out challenge coins
● 30,000 ft view thinker● Spent entire professional career
in Navy (deeply ingrained culture)
Products& Services
- Contextualized, object-oriented database
- Algorithms for processing, analyzing data
- Ability to search for trends across database
- Integration of disparate data sources
- Automation of data analysis
- Improved UX/UI- Predictive notifications- Filtering and layering
features
Customer Jobs
Gains
Pains
Gain Creators
Pain Relievers
- Contextualized, object-oriented database
- Compatible data format
- Incorporate multiple data streams with existing object-oriented database
- Collect & analyze data
- Communicate findings
- Piece together contextualized awareness
- More actionable insights
- Faster identification & response times
- Easy-to-use
- Incorporation of context is manual/mental
- Poor quality / lack of data
- Latency of data -> insight
Analyst (N2)
Value Proposition Canvas
Customer persona:● Sits in front of computer all day● Job is normally boring with bursts
of excitement● Some may have constantly
varying hours/schedules
● “19 year old from Oklahoma”● Regimented schedule● May or may not like what they do
Products& Services
- N/A
- Actually a common operating picture!
- Data is actually synced across databases Customer
Jobs
Gains
Pains
Gain Creators
Pain Relievers
- No hardware to deploy so no risk of asset or personnel loss
- Fewer change orders
- Utilize assets and human capital in order to obtain ISR data on adversary or regions of interest
- Timely and enhanced allocation and deployment of assets
- High manpower, time- Operator error- Safety concern for
deploying in unfriendly territory
- Struggle to redeploy systems on short notice (<12 hours) = frustration
Operations (N3)
Value Proposition Canvas
Customer persona:
● General sense that N2 and N6 “work” for them
● “19 year old from Oklahoma”● Regimented schedule
● May or may not like what they do
(green indicates that validation is still needed)
Map of System Functions and Needs
QUELLFIRE
GCCS (1)
FOBM
STORAGE/COMMS
CST
GCCS (3)GCCS (2)
STORAGE/COMMS
STORAGE/COMMS
Sensors Sensors Sensors
.oth-.json Translator
Visualization
Analytics
Ship-to-Ship Sharing
Long-Term Storage
KEY NEEDSFUNCTIONS
& PROGRAMS
SHIP 2 SHIP 3SHIP 1
Map of System Functions and Needs
QUELLFIRE
GCCS (1)
FOBM
STORAGE/COMMS
CST
GCCS (3)GCCS (2)
STORAGE/COMMS
STORAGE/COMMS
Sensors Sensors Sensors
.oth-.json Translator
Visualization
Analytics
Ship-to-Ship Sharing
Long-Term Storage
KEY NEEDSFUNCTIONS
& PROGRAMS
SHIP 2 SHIP 3SHIP 1
MVP
Storing Historical Data Locally → Less bandwidth usage + ability to do better pattern recognition, alerts
GCCS
MVP (1 week ago)
MVP (1 week ago)
MVP (1 week ago)
Questions?
Customer Workflow
N2
N3
N2(“owns”
the intel)
N3(“owns”
the assets)
Ready-To-Use DataDeployment
Data Acquisition
Data Analysis
Data
Order/Decision
MVP (2 weeks ago)
AIS Weather
MVP (2 weeks ago)
AIS Weather
MVP (2 weeks ago)
AIS Weather
Data Acquisition
ContextualizedDatabase
MVP (3 weeks ago)
Deployment
Last Month
Today
Object-orientedDatabase
Query
- What data is most useful to capture?- What sensor modalities can capture?- What products exist?
- What deployment options exist?- What is easiest to deploy?- What is “good-enough” time to data acquisition?- What is the deployment process?
- Is .kmz format all that is necessary for compatibility?- What do companies like Palantir do today?
Customer Workflow
Customer Discovery (1 week ago)
Hypotheses Experiments Results Action
We want automated data layering
REFINED
- Interviews with CMDR Pablo Breuer (N3/6), CMDR Rob Williams (N0)
- There is too much layering going on- Info overload, esp in Straits of Malacca- Need a better filtering method
- Revamp the MVP to focus on filtering layers- Get a sanitized GCCS screen of Straits of Malacca
A common, easily-searchable database is desireable
VALIDATED
- Interviews with CMDR Rob Williams (N0), CMDR Chris Adams (N3), CTR3 Joseph Baba (N2), CMDR Pablo Breuer (N3/6)- Engagement with Elston ToChip (Palantir)- Interview with Chad Dalton, Pat Kelly (OGSystems)
- Need to submit help ticket to access certain databases- Palantir often interacts with customers who have siloed datasets- 6+ databases that people search through manually - takes hours
- Develop separate MVP to test database functionality- How to establish common database without losing context-specific attributes?- What are the data feeds for 7th Fleet vs. PacFlt vs. PACOM?
Analysts will analyze, no need to do our own algorithms
INVALIDATED
- Interviews with Dr. Evelyn Dahm (SOCPAC advisor, J5/8), CMDR Rob Williams (N0), CMDR Silas Ahn (N2)
- Would be helpful to establish baseline and alert when anomaly occurs- Want push notifications when a ship violates known patterns
- Determine example scenarios that 7th Fleet wants to monitor
Customer Discovery (1 week ago)
Hypotheses Experiments Results Action
We want automated data layering
REFINED
- Interviews with CMDR Pablo Breuer (N3/6), CMDR Rob Williams (N0)
- There is too much layering going on- Info overload, esp in Straits of Malacca- Need a better filtering method
- Revamp the MVP to focus on filtering layers- Get a sanitized GCCS screen of Straits of Malacca
A common, easily-searchable database is desireable
VALIDATED
- Interviews with CMDR Rob Williams (N0), CMDR Chris Adams (N3), CTR3 Joseph Baba (N2), CMDR Pablo Breuer (N3/6)- Engagement with Elston ToChip (Palantir)- Interview with Chad Dalton, Pat Kelly (OGSystems)
- Need to submit help ticket to access certain databases- Palantir often interacts with customers who have siloed datasets- Analysts are spending too much time developing their own custom ETL tools
- Develop separate MVP to test database functionality- How to establish common database without losing context-specific attributes?- What are the data feeds for 7th Fleet vs. PacFlt vs. PACOM?
Analysts will analyze, no need to do our own algorithms
INVALIDATED
- Interviews with Dr. Evelyn Dahm (SOCPAC advisor, J5/8), CMDR Rob Williams (N0), CMDR Silas Ahn (N2)
- Would be helpful to establish baseline and alert when anomaly occurs- Want push notifications when a ship violates known patterns
- Determine example scenarios that 7th Fleet wants to monitor
Customer Discovery (1 week ago)
Hypotheses Experiments Results Action
We want automated data layering
REFINED
- Interviews with CMDR Pablo Breuer (N3/6), CMDR Rob Williams (N0)
- There is too much layering going on- Info overload, esp in Straits of Malacca- Need a better filtering method
- Revamp the MVP to focus on filtering layers- Get a sanitized GCCS screen of Straits of Malacca
A common, easily-searchable database is desireable
VALIDATED
- Interviews with CMDR Rob Williams (N0), CMDR Chris Adams (N3), CTR3 Joseph Baba (N2), CMDR Pablo Breuer (N3/6)- Engagement with Elston ToChip (Palantir)- Interview with Chad Dalton, Pat Kelly (OGSystems)
- Need to submit help ticket to access certain databases- Palantir often interacts with customers who have siloed datasets- People use data analysis products to support preconceived ideas
- Develop separate MVP to test database functionality- How to establish common database without losing context-specific attributes?- What are the data feeds for 7th Fleet vs. PacFlt vs. PACOM?
Analysts will analyze, no need to do our own algorithms
INVALIDATED
- Interviews with Dr. Evelyn Dahm (SOCPAC advisor, J5/8), CMDR Rob Williams (N0), CMDR Silas Ahn (N2)
- Would be helpful to establish baseline and alert when anomaly occurs- Want push notifications when a ship violates known patterns
- Determine example scenarios that 7th Fleet wants to monitor
Customer Discovery (1 week ago)
Hypotheses Experiments Results Action
We want automated data layering
REFINED
- Interviews with CMDR Pablo Breuer (N3/6), CMDR Rob Williams (N0)
- There is too much layering going on- Info overload, esp in Straits of Malacca- Need a better filtering method
- Revamp the MVP to focus on filtering layers- Get a sanitized GCCS screen of Straits of Malacca
A common, easily-searchable database is desireable
VALIDATED
- Interviews with CMDR Rob Williams (N0), CMDR Chris Adams (N3), CTR3 Joseph Baba (N2), CMDR Pablo Breuer (N3/6)- Engagement with Elston ToChip (Palantir)- Interview with Chad Dalton, Pat Kelly (OGSystems)
- Need to submit help ticket to access certain databases- Palantir often interacts with customers who have siloed datasets- JIOC, PacFlt, 7th Fleet do not see the same feeds -> may lag each other by 2-6 hours!
- Develop separate MVP to test database functionality- How to establish common database without losing context-specific attributes?- What are the data feeds for 7th Fleet vs. PacFlt vs. PACOM?
Analysts will analyze, no need to do our own algorithms
INVALIDATED
- Interviews with Dr. Evelyn Dahm (SOCPAC advisor, J5/8), CMDR Rob Williams (N0), CMDR Silas Ahn (N2)
- Would be helpful to establish baseline and alert when anomaly occurs- Want push notifications when a ship violates known patterns
- Determine example scenarios that 7th Fleet wants to monitor