get good data out of bad @nacnudus spreadsheets garmonsway … · 2019. 5. 21. · duncan...

52
Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

Upload: others

Post on 01-Oct-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

Duncan Garmonsway

@nacnudus

Get good data out of bad spreadsheetsWith tidy tools

Page 2: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://twitter.com/albertocairo/status/963133927530483712

Page 3: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://twitter.com/albertocairo/status/963133927530483712

Page 4: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://nacnudus.github.io/underground/

Page 5: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

Uh oh

https://nacnudus.github.io/nzpullover/

Page 6: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 7: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

@sh

arla

gelfa

nd h

ttps:

//tw

itter

.com

/sha

rlage

lfand

/sta

tus/

9967

3912

7562

3915

52

@dv

augh

an32

http

s://b

log.

davi

svau

ghan

.com

/pos

t/tid

ying

-exc

el-c

ash-

flow

-spr

eads

heet

s-in

-r/

Page 8: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://www.brodrigues.co/blog/2018-09-11-human_to_machine/

Page 9: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

http

s://n

acnu

dus.

gith

ub.io

/nzm

igra

tion/

@kuri_pablo http://170.238.108.227/binhnd/RpWebEngine.exe/Portal?BASE=CPVHND2013NAC&lang=ESP

Page 10: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

http

s://g

ithub

.com

/nac

nudu

s/uk

farm

Page 11: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://www.mic.com/articles/160125/pokemon-go-cp-changes-balance-update-hits-every-pokemon-but-onehttps://www.reddit.com/r/TheSilphRoad/comments/5e6jev/silph_research_group_on_cp_balancing/https://docs.google.com/spreadsheets/d/1_WS9FIGSlKVk6XAJTE3TxXIqlBPIQ5Lsx5qifE72vXY/edit?usp=sharing

Page 12: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

http

s://i

nfor

mat

ioni

sbea

utifu

l.net

/vis

ualiz

atio

ns/b

est-i

n-sh

ow-w

hats

-the-

top-

data

-do

g/ v

ia @

Jenn

yBry

an

Page 13: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 14: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 15: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 16: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 17: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 18: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 19: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 20: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 21: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 22: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

filter(row == 3, col == 1)$character

Page 23: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

readr::melt_csv()

googlesheets4::sheets_cells()

tidyxl::xlsx_cells()

Page 24: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 25: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 26: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 27: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 28: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 29: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 30: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 31: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 32: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 33: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 34: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 35: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 36: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 37: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 38: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 39: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 40: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 41: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

1. One-row-per-cell format:

Page 42: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

1. One-row-per-cell format:tidyxlreadrgooglesheets4

Page 43: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

1. One-row-per-cell format:tidyxlreadrgooglesheets4

2. behead() each header in turn

Page 44: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

1. One-row-per-cell format:tidyxlreadrgooglesheets4

2. behead() each header in turnunpivotr

Page 45: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 46: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

Via

Dav

id R

obin

son

and

Jenn

y B

ryan

http

s://r

pubs

.com

/dgr

two/

tidyi

ng-e

nron

Page 47: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 48: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

((IF((103-B$89)=103,0,(103-B$89)))+(IF((200-B$95)=200,0,(200-B$95)))+(IF((196-B$98)=196,0,(196-B$98)))+(IF((200-B$101)=200,0,(200-B$101)))+(IF((70-B$104)=70,0,(MIN(40,(70-B$104))))+(IF((78-B$109)=78,0,(MIN(50,(78-B$109)))))+(IF((103-B$114)=103,0,(MIN(66,(103-B$114)))))+(IF((195-B$119-B$124-B$129-B$134-B$139)=195,0,(MIN(70,(195-B$119-B$124-B$129-B$134-B$139)))))+(IF((64-B$144)=64,0,(MIN(50,(64-B$144)))))+(IF((48-B$149)=48,0,(MIN(20,(48-B$149)))))+(IF((44-B$154)=44,0,(MIN(20,(44-B$154)))))+(IF((130-B$159)=130,0,(MIN(20,(130-B$159)))))))

Page 49: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 50: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools
Page 51: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

https://rstudio.cloud/project/269487

Page 52: Get good data out of bad @nacnudus spreadsheets Garmonsway … · 2019. 5. 21. · Duncan Garmonsway @nacnudus Get good data out of bad spreadsheets With tidy tools

My stuff:https://github.com/nacnudus/tidyxlhttps://github.com/nacnudus/unpivotrhttps://nacnudus.github.io/spreadsheet-munging-strategies/https://rstudio.cloud/project/269487

Related:https://github.com/luisDVA/unheadrhttps://github.com/ianmoran11/tidyABShttps://databaker.sensiblecode.io/

Duncan Garmonsway

@nacnudus

Get good data out of bad spreadsheets