nowcasting food prices using social media · 2015-10-27 · nowcasting food prices using social...
TRANSCRIPT
![Page 1: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/1.jpg)
Nowcasting Food Prices using Social Media
Jong Gun Lee
Data Scientist
UN Global Pulse
![Page 3: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/3.jpg)
Nowcasting Food Prices in Indonesia using Social
Media Signals
http://unglobalpulse.org/nowcasting-food-prices
![Page 4: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/4.jpg)
4
![Page 5: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/5.jpg)
A Generative Model
Hypothesis - “Today’s price P(i+1) is inferable
from yesterday’s price P(i) and today’s tweet price Ptweet(i+1)”
![Page 6: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/6.jpg)
A Generative Model
Hypothesis - “Today’s price P(i+1) is inferable
from yesterday’s price P(i) and today’s tweet price Ptweet(i+1)”
Components for weighing and filtering as well as refreshing
![Page 7: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/7.jpg)
Data
• Primary data
– About 23,000 tweets for the following four commodities
• Beef, Chicken, Chili, and Onion
– Taxonomy
• Commodity name & Price (number & currency)
• Unit, if available
• Secondary data
– Daily price information from Indonesian Ministry of Trade
• Duration
– 16 months between 2012/07 and 2013/10
![Page 8: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/8.jpg)
Beef (daging sapi)
• 5,707 tweets used among 14,473 tweets
• Pearson correlation - 0.87
![Page 9: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/9.jpg)
Chicken (daging ayam)
• 974 tweets used among 5,223 tweets
• Pearson correlation - 0.81
![Page 10: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/10.jpg)
Onion (bawang)
• 610 tweets used among 1,954 tweets
• Pearson correlation - 0.85
![Page 11: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/11.jpg)
Chili (cabe)
• 206 tweets used among 1,772 tweets
• Pearson correlation - 0.76
![Page 12: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/12.jpg)
NOWCASTING FOOD PRICES (ONGOING)
![Page 13: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/13.jpg)
Google Trends Data
• Google Trends service provides normalized, aggregated search
volume per day per keyword up to provincial level.
![Page 14: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/14.jpg)
A Simple Linear Model
“Provincial Price = Past National Price + Recent Google Trends in Province ”
Pi = α + β1 MoTi-2 + β2 MoTi-4 + β3 GTi + β4 GTi-1 + β5 GTi-3
MoTi-1: National price two weeks ago MoTi-4: National price four weeks ago
GTi: Google Trends this week GTi-1: Google Trends last week
GTi-3: Google Trends three weeks ago
Rice Price in Central Java 2014
![Page 15: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/15.jpg)
New Digital Data Sources + Reports by Citizens?
![Page 16: Nowcasting Food Prices using Social Media · 2015-10-27 · Nowcasting Food Prices using Social Media Jong Gun Lee Data Scientist ... –16 months between 2012/07 and 2013/10. Beef](https://reader034.vdocuments.mx/reader034/viewer/2022042713/5fb0562cb20c8117077e24a1/html5/thumbnails/16.jpg)
www.unglobalpulse.org
@unglobalpulse
@pulselabjakarta
@pulselabkampala