pdr for lab-forming field and its benchmarking
TRANSCRIPT
PDR for lab-forming field and its benchmarking
Takeshi Kurata12, Ryosuke Ichikari1,Tatsuro Myokan12, Masakatsu Kourogi1, Katsuhiko Kaji3, and Nobuo Kawaguchi4
1AIST, 2Univ. of Tsukuba, 3Aichi institute of technology, 4Nagoya Univ.
ISMAR 2016 Workshop: Standards for Mixed and Augmented Reality (2016/9/23)
Lab-forming Field & Field-forming Lab
• Borrowing from “Terraforming”• Lab-forming Field: Transforming a real field into a lab-like place. • Field-forming Lab: Transforming a laboratory into a field-like place.
2
Big Data on Result/Behavior/Environment for Lab-forming Field
3
Case studyin Japanese Restaurant “Ganko”
• Objectives1. (for AIST) to test the CSQCC
(Computer-Supported QCC) suites in a real service field.
2. (for the restaurant) to observe effects of process improvement planned by CSQCC.
• Place– Japanese cuisine restaurant
GANKO Ginza 4-chome (Tokyo)
• Term– 1st term
• January 12 to 18, 2011– 2nd term
• February 3 to 9, 2011
4
Dining area Course dishes
1st term(Jan. 12-18, 2011)
for observing ordinary operations
QC circlefor making improvement plans
2nd term (Feb. 3-9, 2011)for observing improved operations
5 B2
B1
Dinning Area
Kitchen
Office room
Pantry
During Discussion in CSQCC6
Trajectory of a wait staff in lunch time: 12:00-14:00
Fact: Going in and out of the kitchen/office to no small extent.Possible result: Difficulty in concentrating on guest service.Cause: Cell phone everywhere, but reservation book only in the office room.Possible improvement: e-reservation book
Dinning Area
Kitchen
Office room
T. Fukuhara, R. Tenmoku, T. Okuma, R. Ueoka, M. Takehara, and T. Kurata, "Improving Service Processes based on Visualization of Human-behavior and POS data: a Case Study in a Japanese Restaurant“, ICServ2013, pp.1-8.
PDR(Pedestrian Dead-Reckoning)Estimates velocity vector, relative altitude, and action type by measurements from a wearable sensor module.
Wearing a sensor module on waist (2D SHS (Steps and Heading Systems) PDR) Easy to wear and maintain Easy to measure data for action recognition Relatively easily apply for handheld setting compared to shoe-mounted PDR
(3D-INS (Inertial Navigation System) PDR)
7
Handheld PDR From PDR to PDRplus
10-axis sensors• Accelerometers• Magnetic sensors• Gyro sensors• Barometer
Shoe-mounted PDR
Waist-worn PDR
PDR Business Launched in Japan
Collaboration among NTT Docomo, ZDC, AKM, and AIST.
A PDR driven Indoor navigation mobile app was released in Apr, 2015. (As of Sep, 2016,
440 areas including subways and underground shopping arcades all across
Japan)
Collaboration between MegaChips and AIST.
Sensor Hub LSI “Frizz” was released in Jan, 2015 and MegaChips plans to sell 10 million Frizz products in 2015. (Power-aware motion
coprocessor optimized for PDR)
8
• M. Kourogi and T. Kurata, Personal positioning based on walking locomotion analysis with self-contained sensors and a wearable camera, ISMAR2003, pp. 103–112, (2003).
AR by PDR + Image registration(1999-2003)
Panorama-based Annotation: IWAR1999, ISWC2001,
ISMAR2003
G
Environmental mapA
B C D
E
A
B
C
F
Input frames
Position at whicha panorama is taken
PositionDirection
235 [deg]
5 [deg]From the user’s camera
Located Orientated
9
Frontier of PDR: Walking direction estimation
10
• Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.
Frontier of PDR: Walking direction estimation
11
• Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.• Long Paper: Christophe Combettes, Valerie Renaudin, Comparison of Misalignment
Estimation Techniques Between Handheld Device and Walking Directions, IPIN 2015.• FIS was proposed by Kourogi and Kurata in PLANS 2014.
“Globally, the FIS method provides better results than the other two methods.”
Frequency analysis of Inertial Signals
Forward and Lateral Acc. Modeling
Principal Component Analysis
Overview: History of our PDR12
ISWC2001
IWAR1999
ISMAR2003
PLANS2014
PLANS2010 ICServ2013
Docomo map navi
Image registration + Gyro
Panorama-based annotation (Image-registration-based positioning)
Image registration + PDR
PDRplus (PDR + Action recognition)
Handheld PDR(Walking-direction estimation)
2015- 2015-
PDR module
2011-
Academia
Industry
Before PDR
ICAT2006 PDR + GPS + RFID
In the year of 2010• iPhone 4: the first popular consumer mobile device
equipped with 9-axis sensors including accelerometers, magnetic sensors, and gyro sensors
13
G-spatial EXPO 2010:Handheld PDR on iPhone 4(World’s first-ever live demo)
PLANS2010, PLANS2014
Global Trend on PDRPDR R&D players have rapidly indicated their presence all over the world on and after 2010.
Movea (France)
Sensor Platforms (USA)
CSR (UK)
TRX (USA)
Trusted Positioning (Canada)
14
Acquired by QualcommAcquired by InvenSenseAcquired by InvenSense
Acquired by Audience
Indoo.rs (USA)
SFO
Standardization on PDR Benchmarking• PDR related R&D is highly active worldwide: Necessity for sharing
common measures.• Description of the performance should be unified in spec sheets
and scientific papers.• Different measures from absolute positioning methods such as
GNSS, Wi-Fi, and BLE are required for PDR, which is a method of relative positioning.
• PDR Benchmark Standardization Committee was established in 2014 as a platform of the grassroots activity.
15
https://www.facebook.com/pdr.bms
Support Organizations• Asahi Kasei Corporation, Asia Air Survey Co., Ltd. (Y. Minami), INTEC Inc.,
MTI Ltd., KDDI R&D Laboratories, Inc., KOKUSAI KOGYO CO., LTD.,SHIBUYA KOGYO CO., LTD., Koozyt, Inc., SITESENSING, inc., Sharp Corporation, Sugihara Software and Electron Industry Co., Ltd. (SSEI), Information Services International-Dentsu, Ltd. (ISID), Hitachi, Ltd.,Frameworx, Inc. (S. Watanabe), MULTISOUP CO.,LTD., Milldea, LLC, Murata Manufacturing Co., Ltd., MegaChips Corporation, Recruit Lifestyle Co., Ltd. (K. Ushida), RICOH COMPANY, LTD., Rei-Frontier Inc.,
• Aichi Institute of Technology (K. Kaji), Akashi National College of Technology (I. Arai), Kanagawa Institute of Technology (H. Tanaka), Keio University (S. Haruyama, N. Kohtake, M. Nakajima), University of Tsukuba (T. Kurata), Tokyo Institute of Technology (S. Okada), Nagoya University (N. Kawaguchi), Niigata University (H. Makino), Ritsumeikan University (N. Nishio), National Institute of Advanced Industrial Science and Technology (AIST) (T. Kurata, M. Kourogi), Human Activity Sensing Consortium (HASC), Location Information Service Research Agency (LISRA)
• 33 organizations in Japan as of April, 2016
16
Activity Examples in Japan17
1st meeting of the committee (2014/05/22)
1st seminar in LBJ (2014/06/11)
2nd meeting of the committee (2015/05/19)
2nd seminar in LBJ (2015/06/13)
Organized session in HCG symposium (2014/12/17)
3rd seminar in LBJ (2016/06/10) LBJ: Location Business JapanHCG: Human Communication Group
Organized session in HCG symposium (2015/12/16)
3rd meeting of the committee (2015/05/19)
Scene in data collection19
Multi-Algorithm On-Site Evaluation System• Evaluates the accuracy of each PDR algorithm automatically as often
as sensor data is uploaded to the server• Provides trajectory images so that participants can compare their PDR• algorithms in real time.
20
http://pdrsv.hasc.jpK. Kaji, K. Kanagu, K. Murao, N. Nishio, K. Urano, H. Iida, N. Kawaguch, Multi-Algorithm On-Site Evaluation System for PDR Challenge, ICMU2016, (to appear)
UbiComp/ISWC 2015 PDR Challenge Corpus• Is now open to the public. (http://hub.hasc.jp/)
21
Routes 5
Devices 7
Subjects 93
# of pedestrian sensing data 241
# of pedestrian sensing data with calibration data 230
# of pedestrian sensing data with LIDAR data 10
Avg. of walking time [sec] 101
Avg. of moving distance [m] 115
Avg. of angular change [°] 606
K. Kaji, M. Abe, W. Wang, K. Hiroi, and N. Kawaguchi, UbiComp/ISWC 2015 PDR challenge corpus, HASCA2016 (UbiComp2016 Proceedings: Adjunct), pp.696-704
Statistics of the corpus
Detailed route statistics of pedestrian sensing data with calibration data
Open Data Contest in Logistics &PDR Challenge in Warehouse
• Open data contest in logistics by Frameworx– Submission: 2016/4/18-2016/7/18– Award ceremony: 2016/9/12
• PDR Challenge in Warehouse– Now planning– Will be held as a Japanese
domestic contest in 2016– Will be held as an international
contest in IPIN 2017
22
PDR Challenge Series• Ubicomp/ISWC 2015 PDR Challenge
– Scenario: Indoor Navigation– On-site– Continuous walking while keeping watching the navigation
screen by holding the smartphone– Several minutes per trial
• 2016/2017 PDR Challenge in Warehouse (tentative)– Scenario: Picking work in a warehouse– Off-site– Not only walking but various actions including picking and
carrying– Several hours per trial– Will be held in IPIN 2017
23
Examples of picking workers’ trajectories estimated by PDR + WMS
(Warehouse Management System)
24
How to design benchmark Indicators?
• Other aspects to be considered– Reliability: Different measures from absolute positioning methods
are required for PDR– Efficiency: Power consumption– Repeatability: Temperature Hysteresis, Magnetic field, etc.– Representativeness: How to hold, Route shape, etc.
25
Benchmark indicators of vision-based spatial registration and tracking for MAR (ISO/IEC WD 18520)
How to compare and visualize?26
Easy Difficult
Method 1
Easy Difficult
Method 2
How to compare and visualize?27
Easy Difficult
Met
hod
1M
etho
d 2
Competitions: IPIN and the others(cf. EvAAL presentation in IPIN 2105 etc.)
28
IPIN year EvAAL, IPSN, UbiComp/ISWC
Zurich, Switzerland 2010 universAAL is launchedGuimaraes, Portugal 2011 EvAAL: indoor localization
Sidney, Australia 2012 EvAAL: + activity recognitionMontbeliard, France 2013 EvAAL: same as 2012
Busan, Korea1st IPIN competition 2014 EvAAL: 3 floors, smartphone
IPSN: infrastruc. based + free
Banff, CanadaEvAAL-ETRI comp. 2015
EvAAL-ETRI: 6 floors, on/off-siteIPSN: infrastruc. based + free
UbiComp/ISWC: 2 floors, smartphone PDR, 90 subjects
Madrid, SpainIndoor Localization
Competition2016 IPIN: smartphone (on/off-site), PDR, Robot
IPSN: infrastruc. based + free, 2D/3D
IPIN2017
29
ISMAR2015
Thank you!• AIST is now hiring for Tenure-track, Postdoc, and
RA (PhD) positions at Tsukuba, Japan.• Target research fields are ↓ ↓ ↓
30