![Page 1: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/1.jpg)
Mining Social Media in Extreme Events: Lessons Learned from the
DARPA Network Challenge
Nicklaus A. Giacobe, Hyun-Woo “Anthony” Kim and Avner Faraz
[nxg13, hxk263] @ist.psu.edu, [email protected]
![Page 2: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/2.jpg)
The DARPA Network Challenge
• Conceived to better understand and harness the power of the Internet– How does one get a video to go viral on
YouTube?– Leveraging social networks to solve
“intractable” or “impossible” tasks– Short time frame
• Announced on October 29, 2009• Challenge occurred on December 5, 2009 (5 weeks
of possible prep time)
![Page 3: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/3.jpg)
The DARPA Network Challenge
• DARPA launched 10 red weather balloons• Tethered to fixed locations in public places
across the continental United States• Find them and report GPS coordinates of
all 10 before anyone else does and win the $40,000 prize
• The balloons would be “up” from 0900 EST until 1600 local time (7-10 hours) and then taken down
![Page 4: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/4.jpg)
![Page 5: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/5.jpg)
Final Results
• 4600 “Teams” registered with DARPA• 58 Teams were “in the hunt” and submitted
2 or more correct locations• Top teams used mass marketing and
mass motivation techniques (offered to share the prize $ with observers)
• Each team has a “lesson” applicable to homeland security
![Page 6: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/6.jpg)
Top Finalists – We’re #10!
From: https://networkchallenge.darpa.mil/FinalStandings.pdf
![Page 7: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/7.jpg)
Overview
• Who are the ?• Caucus of academic institutions studying information
science• The iSchools are interested in the relationship between
information, people and technology. • There are 28 academic Institutions (Colleges or
Departments) in various universities across 8 countries• Some iSchools have roots in Library Science,
Computational Sciences, MIS, Business Management, Cognitive Science, Human-Computer Interaction (HCI) and other fields. Each iSchool has its own focus and competencies.
![Page 8: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/8.jpg)
![Page 9: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/9.jpg)
Team Organization
• Command Structure– Attempted to follow ICS from Fire Service– Most Team Members unfamiliar with this
organizational structure, therefore very limited success
• Operational Section– 2 Branches – Direct Observation and Cyberspace
Search
• Facilities– 211 IST – EEL – Command Post– 208 IST – Classroom – Cyberspace Search
![Page 10: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/10.jpg)
![Page 11: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/11.jpg)
![Page 12: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/12.jpg)
![Page 13: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/13.jpg)
Methods
• Direct Observation– Recruit observers from the iSchools Caucus– Report sightings through
• Website• Phone / SMS• Email
• Cyberspace Search– Multiple Intel Teams searching open communications
online• Twitter, Competitor Websites, No Hacking
• Confirmation and Decision Making
![Page 14: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/14.jpg)
Methods
• Technologies Used– Twitter Capture – Anthony Kim– Custom Crawler – Madian Khabsa– Maltego2 – Avner Faraz
![Page 15: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/15.jpg)
Methods
Direct Observation•Recruiting•All of
iSchools to send out recruiting messages
•Phone/SMS/Web/Email Reporting
Cyber Search / Intel• Twitter• Websites• Competing Team Sites
Observer Confirmation• Pre-Registered Observers• Recruited Observers
• iSchools Network• “Friends” Network
Analysis•Photo
Analysis•Provenance
Evaluation
D-S Evidence Theory
Combination
![Page 16: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/16.jpg)
Methods
• Dempster-Shafer Evidence Theory Combination– Combine evidence from multiple sources under
uncertainty– Apply confidence weights to sensor data
• Intended, but applied cognitively– Analysts were to provide report data with confidence
values (0=low, 10=high)– Some algorithmic process would have been needed
to combine large numbers of reports– … but we had *extremely low* # of reports
![Page 17: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/17.jpg)
Pre-Challenge Org.
![Page 18: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/18.jpg)
Team Organization
• Other Universities– Various Schools to send recruiting messages– UNC – distributed own phone number and
email address for their recruiting messages– Univ. of Illinois – single-person cyberspace
search division
![Page 19: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/19.jpg)
![Page 20: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/20.jpg)
Team Organization
• Finance and Logistics– Google Voice (814-4BALL01)
– Website Design (balloon.ist.psu.edu)
– Email Address ([email protected])
– Google Wave (Intel and Command Comms)
– Private Twitter (late attempt at outbound comms)
– Coffee, Donuts, Pizza, Soda and Homemade Cookies ($100!)
– Incentives and Rewards (10 GPS Systems Offered by iSchools)
![Page 21: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/21.jpg)
Team Members
• University of Illinois– John Unsworth– Maeve Reilly– Karyn Applegate
• University of North Carolina– Aaron Brubaker– Kjersti Kyle
• Other iSchools– Marketing/Communications Staff
![Page 22: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/22.jpg)
Team Members
• Penn State University– Command Post Team
• Nick Giacobe• Wade Shumaker• Louis-Marie Ngamassi Tchouakeu• John Yen• Jon Becker (p/t)• Michelle Young (p/t)
– Logistics / Website• Shannon Johnson• Lei Lei Zhu
![Page 23: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/23.jpg)
Team Members
• Penn State University– Operations (Cyberspace Search Branch)
• Crawler Task Force– Madian Khabsa and Jian Huang
• Twitter Capture Task Force– Hyun Woo “Anthony” Kim and Airy Guru
• Intelligence Analysts– Chris Robuck– Greg Traylor– Anthony Maslowski– Gregory O’Neill– Joe Magobeth– Avner Faraz– Matt Maisel– Earl Yun
![Page 24: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/24.jpg)
Results
• Recruiting Message Reports– University of Illinois – Email to iSchool alumni; to faculty, staff and
students.– University of Pittsburgh – Email to all alumni; to faculty, staff and
students; Webpage article on main page & alumni news page; LinkedIn announcement to iSchool at Pitt group members; and Facebook announcement to iSchool at Pitt group members. Alumni email distributed to 4,674 alumni. 114 opened the DARPA link. Email blasts to 936 students 41 faculty.
– Penn State – 338 fans on Facebook and 377 followers on Twitter. Re-tweets, questions about the Challenge on Twitter, but no activity on Facebook. 2,125 alumni received the e-mail.
– UCLA – Sent messages to faculty, staff, students and alumni.– Drexel – Alumni listserv, Facebook, Tweeter electronic newsletter for
undergrads and grads, online learning system grad web site release.
![Page 25: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/25.jpg)
Results
• Direct Reporting Data– 1 Report From Observer– 1 Pre-recruited Observer tasked for confirmation– 8 Observers recruited for confirmation
• Website Hit Data– 567 hits (Terrible!)
• 16 Case Studies of Individual Balloon Reports…
![Page 26: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/26.jpg)
Overview of Cases
Erie, PA
Competitor Site
Observer
Albany, NY
Twitter w/ picture
Observer/Photo Analysis
Royal Oak, MI
Twitter w/ picture
Observer - Fake
Providence, RI
Twitter w/picture
Observer/Photo Analysis
Seattle, WA
Twitter w/o Picture
Observer
Champaign, IL
Competitor Site
Observer
Des Moines, IA
Self Recant
Christiana (Glascow), DE
Twitter w/ Photo
Detail /Photo Analysis
Bithlow , FL
Custom Cralwer
Observer
Charlottesville, VA
Observer Report
Conf.Details / Call Back
Scottsdale, AZ
Competitor Site
Detail /Photo Analysis
Portland, OR
Twitter w/ Picture
Observer
San Francisco, CA
Twitter/Blog
Detail/Photo Analysis
Santa Barbara, CA
Detail/Photo Analysis
Westfield, NJ
Twitter w/o Picture
Conflicting Data
Memphis , TN
Competitor Trade Offer
Never Confirmed
![Page 27: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/27.jpg)
Results: Case Study 1
• Location: Erie, PA• Original Report: 10balloons.com website• Method: Observer mobilized• Notes:
– Early report – 9:30AM– Evaluated whether 10balloons.com website
was going to be useful intel or not– Observer was not pre-registered, was
personal friend of command post personnel
![Page 28: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/28.jpg)
![Page 29: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/29.jpg)
![Page 30: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/30.jpg)
![Page 31: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/31.jpg)
![Page 32: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/32.jpg)
![Page 33: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/33.jpg)
![Page 34: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/34.jpg)
Results: Case Study 2
• Location: Albany, NY• Original Report: Twitter Feed• Method: Observer mobilized• Notes:
– Observer provided convincing photo evidence – no balloon at this location
– Subsequent photo analysis confirmed this was a manufactured image
![Page 35: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/35.jpg)
Results: Case Study 2
• Evidence for– Reputability (3/10)
• Established Twitter Acct
– Content (6/10)• Photo of a balloon• Balloon Number 6?• Weather – match • Location – exact coords
not provided, but
discoverable
• Evidence Against– Reputability (9/10)
• Pre-recruited observer• Known person (PSU
Alumni Assn chapter president)
– Content (10/10)• Excellent quality photo• Same angle, confirming
coincidental landmarks/features
![Page 36: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/36.jpg)
![Page 37: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/37.jpg)
![Page 38: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/38.jpg)
![Page 39: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/39.jpg)
![Page 40: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/40.jpg)
![Page 41: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/41.jpg)
![Page 42: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/42.jpg)
Results: Case Study 3
• Location: Royal Oak, MI• Original Report: Twitter• Method: Observer Mobilized• Notes:
– Observer went to the location and talked to the store owner who admitted that the balloon was a “publicity stunt” – knew about DARPA challenge and put up own balloon.
– Observer was not pre-recruited
![Page 43: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/43.jpg)
Results: Case Study 3
• Evidence for– Reputability (3/10)
• Established Twitter Acct
– Content (4/10)• Photo of a balloon• No “DARPA” pennant ,
not balloon 6• Weather – match for location• Location – Had GPS
coordinates for location from photo
• Evidence Against– Reputability (9/10)
• Observer recruited after the fact
– Content (8/10)• No photos sent• Good report, verbal
description of events
![Page 44: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/44.jpg)
Results: Case Study 4
• Location: Providence, RI• Original Report: Twitter• Method: Observer Mobilized• Notes:
– Observer was a friend of one of the analysts in the Intel Division – reported no balloon at that location
– Photo analysis provided repeat of the exact fabricated image
![Page 45: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/45.jpg)
Results: Case Study 4
• Evidence for– Reputability (3/10)
• Established Twitter Acct
– Content (4/10)• Photo of a balloon
• Evidence Against– Reputability (9/10)
• Observer recruited after the fact
– Content (8/10)• No photos sent• Good report, verbal
description of location
– Photo Evaluation (10/10)• Reproduced shop job• Edge evaluation
![Page 46: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/46.jpg)
![Page 47: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/47.jpg)
![Page 48: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/48.jpg)
Results: Case Study 5
• Location: Seattle, WA• Original Report:• Method: Observer Mobilized• Notes:
– Report was for balloon over the University of Washington Library
– Observer was Assoc. Dean of LIS College at UW
![Page 49: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/49.jpg)
Results: Case Study 6
• Location: Champaign, IL• Original Report: 10Balloons.com• Method: Observer Mobilized• Notes:
– No picture– Known observer/Team member in close
proximity to this location
![Page 50: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/50.jpg)
Results: Case Study 7
• Location: Des Moines, IA• Original Report: Twitter• Method: Self-Recant• Notes:
– Tweet gave location/street address of reporters home
– No photo evidence– Reporter then re-tweeted complaining about
people running through her yard and then recanted
![Page 51: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/51.jpg)
Results: Case Study 8
• Location: Christiana, DE• Original Report: Twitter w/ coordinates• Method: Photo/Details Analysis• Notes:
– Photos didn’t match Google Maps photos – site was under construction
– Called YMCA Across street from site to confirm
![Page 52: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/52.jpg)
Results: Case Study 9
• Location: Bithlo, FL• Original Report: Crawler• Method: Observer mobilized• Notes:
– Intel team member had a contact in the area– Reported that there was no balloon in that
location
![Page 53: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/53.jpg)
Results: Case Study 10
• Location: Charlottesville, VA• Original Report: Observer Submission• Method: Re-contact observer for details• Notes:
– Alternate observer not available in time– Primary observer re-dispatched to collect
DARPA paperwork– Additional confirmation through intel channels
![Page 54: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/54.jpg)
Results: Case Study 11
• Location: Scottsdale, AZ• Original Report: Fark.com (Competitor
Site)• Method: Extensive Detail Analysis• Notes:
– Interesting story– Unique username– Extensive investigation of reporter’s address, phone
number, etc in online records led to location– Poor attempt at deception– Photographic evidence confirmed location
![Page 55: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/55.jpg)
Results: Case Study 11
• A forum had a member who spotted a red balloon near his house. The post was shortly deleted but his user name was Mini Ditka.
• Using only his username and e-mail address on the forum, I tried using Maltego to perform an OSINT trace on the e-mail address using a transform I had coded for maltego last year.
• The analysis pulled his real name (Scott Shepherd) facebook, linkedin, flickr, spock and Myspace profiles and was able to match it to that e-mail address.
• I further analyzed the subject and was able to find possible phone numbers and web-sites he might have owned.
• In the end: We surmised that the balloon must have been in the Scottsdale area using only his e-mail and username. The actual location was found using image analysis by other team members.
![Page 56: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/56.jpg)
OSINT on E-mail address from deleted post @ forum
FacebookFlickrMyspaceLinkedin
![Page 57: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/57.jpg)
Total Network View For Scott Shepherd
E-mails, Phones, Social Networks, Web-sites, Blogs, Address hits
![Page 58: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/58.jpg)
Phone Number Mining on Scott Shepherd
![Page 59: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/59.jpg)
Results: Case Study 12
• Location: Portland, OR• Original Report: Twitter Feed• Method: Observer mobilized• Notes:
– Photo matched location, gross coordinates, but wanted more details
– Observer confirmed location and provided• Additional photographic evidence• Photo of certificate
– Doctored photo of certificate circulated in the channel by others later
![Page 60: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/60.jpg)
![Page 61: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/61.jpg)
![Page 62: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/62.jpg)
![Page 63: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/63.jpg)
Verify Location Using Google Earth
![Page 64: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/64.jpg)
Results: Case Study 13
• Location: San Francisco, CA• Original Report: Twitter/Blog• Method: Photo evidence• Notes:
– No observer available– Established Twitter Acct/Blog– Multiple photos/angles– GPS Data provided, matched
building/features on Google Maps
![Page 65: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/65.jpg)
![Page 66: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/66.jpg)
![Page 67: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/67.jpg)
![Page 68: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/68.jpg)
Results: Case Study 14
• Location: Santa Barbara, CA• Original Report: Twitter• Method: Photo Evidence• Notes:
– High quality picture available– Unlikely photoshop job based on other photos
available– Location report matches pictures of location
on Google maps
![Page 69: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/69.jpg)
Santa Barbara, CA
![Page 70: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/70.jpg)
Results: Case Study 15
• Location: Westfield, NJ• Original Report: Twitter• Method: Conflicting Details with Known
Good Data• Notes:
– Balloon number conflicted with AZ data that we had high confidence in (both reported as Balloon #2)
– No additional information /no photo– Discussion/disagreement on whether the location for
this balloon was right or not
![Page 71: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/71.jpg)
Results: Case Study 16
• Location: Memphis, TN• Original Report: Competitive Team/Trade• Method: Unable to confirm• Notes:
– Email to [email protected] from a competitor
– Claimed to have location in TN– Unreliable Trade/info– Trading coordinates with other teams was not
part of our plan/strategy
![Page 72: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/72.jpg)
Results: Others
• Locations: – Katy, TX– Miami, FL– Atlanta, GA
• No evidence was found by our team regarding these locations
![Page 73: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/73.jpg)
Summary of Results
• All 15 cases analyzed with correct assessment
• Dispatched observers to 9 locations
• 5 true locations
• 9 false locations
• 1 sighting reported to us directly
Direct Observation
Cyberspace Search
AnalysisObserver Confirmation
![Page 74: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/74.jpg)
Social Media in Extreme Events
• Information Source (Tweets + Geotag + Image) together with GoogleMaps
• Disturbance / Deception• Crowd Sourcing
![Page 75: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/75.jpg)
Tweets/hour (EST)
Dec 04 Dec 05 Dec 06 Dec 07 Dec 08 Dec 09
![Page 76: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/76.jpg)
Tweets/30min on the day of the challenge(Total: 6813 tweets)
A.M. P.M.
Winner Announc
ed
Ads.
Balloons Launched
![Page 77: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/77.jpg)
Three Uses of Tweets
1) Information Source (Tweets + Geotag + Image) together with GoogleMaps
2) Disturbance
3) Crowd Sourcing
![Page 78: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/78.jpg)
(1) Tweets as Information Source
• Tweets with Geotags– Currently only from Twitter Clients for iPhone – 39 out of about 20,000
Images from http://www.twitter-360.com/
![Page 79: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/79.jpg)
Image from http://img129.yfrog.com/i/nhhn.jpg/
“Spotted DARPA balloon #1 in this
very central location”
Tweets as Information Source - Example 1
![Page 80: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/80.jpg)
(1) Information Source - Example 1
Map from maps.google.com
Spotted DARPA balloon #1 in this
very central location.
(37.7879,-122.4073 ) Union Square
San Francisco, CA 94108
![Page 81: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/81.jpg)
(1) Information Source - Example 2
Map from maps.google.com
“Red balloon siting in Marina Del Rey,
CA #DARPA”
( 33.9741,-118.4317 ) Pacific Coast Hwy
Los Angeles, CA 90094
![Page 82: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/82.jpg)
Image from http://twitpic.com/s9k7a
Information Source - Example 2
![Page 83: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/83.jpg)
(2) Tweets as Disturbance Means
![Page 84: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/84.jpg)
(3) Tweets as Crowd Sourcing Means by the MIT team
A.M. P.M.
Help MIT Teamat balloon.media.mit.edu
Balloons Launched
Balloons began to
be put down
Winner Announc
ed
![Page 85: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/85.jpg)
Conclusions
• Open source and social media provide a rich source for gathering and fusing real-time information.
• The Challenge is a nation-wide experiment that is well aligned with the college’s strategic research regarding Extreme Events and Web Science.
• Data and experience gathered are incorporated into SRA courses to enhance the curriculum.
• Integrate social science and STEM education for K-12 using projects related to extreme events.
• NSF project on developing infrastructure for classifying, geo-tagging Tweets/messages for disaster relief at Haiti.
![Page 86: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/86.jpg)
Competitive Analysis
• MIT (10/10)– Market incentive, strong brand, strong pre-
contest press, Ultimately the Winner• Georgia Tech (9/10)
– Red Cross Donation, 2nd place– Why weren’t they part of our team, or us part
of theirs?• Groundspeak Geocachers (7-8/10)
– Market + Donation model– The “right” sensor network (?)
![Page 87: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/87.jpg)
Discussion
• Lessons Learned– Start Earlier– Motivate Observers (promise of cash seemed to work)
– Pick the “right” observer network– Communicate often to maintain motivation– Use OS Intel – it works and it’s cheap!– Be ready to weed out fabricated data
• Photoshop jobs are easy to do• Deceiving pictures are hard to refute
– Define and clarify roles/expectations early and often
![Page 88: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/88.jpg)
Future Work
• Automatic Classification of Microblog Data– Hyun-Woo “Anthony” Kim’s poster yesterday -
he’s here today as well, please see him after• Entity Extraction from Microblog Data
– Useful “entities” in this Challenge• Data Fusion Methods for “soft” data
– Higher levels of inference– Difficult to fuse textual information
![Page 89: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/89.jpg)
DARPA’s Intentions
What did DARPA intend? http://www.youtube.com/watch?v=P_hjpva8gBM (11:15)
When the problems are great, the tendency for each of us is to step away because we believe, even hope that someone else, perhaps smarter, perhaps with more resources will solve those very difficult problems. But I have found that those imaginary people do not exist. There isn’t someone else. It is people just like you and me… There are no imaginary people to do this work. There is no backup plan.
Regina Dugan, Director of DARPA
![Page 90: Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge](https://reader035.vdocuments.mx/reader035/viewer/2022062217/56813532550346895d9c9890/html5/thumbnails/90.jpg)
Thank You!
• Questions?
• QR Code to my website: