strip your text fields - exeter web feb/2016
TRANSCRIPT
![Page 2: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/2.jpg)
about• Data Engineer• @Crowdcube
![Page 3: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/3.jpg)
uses• serialised Arrays• JSON• large strings• images*
![Page 4: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/4.jpg)
![Page 5: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/5.jpg)
how does it work?
![Page 6: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/6.jpg)
alter table•creates a temporary table with the new structure
•copies the data from the old table to the new one
•consolidates the new table
![Page 7: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/7.jpg)
example
![Page 8: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/8.jpg)
alter PK: INT -> BIGINT
- 2.147.483.648 TO
2.147.483.647
INT
- 922.337.203.854.775.808 TO
922.337.203.854.775.807
BIGINTIllus
trat
ion
out o
f sca
le
![Page 9: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/9.jpg)
case• > 750 GB• > 380 million lines• 3 TEXT fields• Auto increment: 898.191.090
![Page 10: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/10.jpg)
how long did it take?
![Page 11: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/11.jpg)
2 days
![Page 12: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/12.jpg)
why?
![Page 13: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/13.jpg)
speed
fast slow
![Page 14: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/14.jpg)
storage• 1 file in the disk for each row for each TEXT
field• stored in different location than the table
data itself• each field up to 4 MB
![Page 15: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/15.jpg)
engines
![Page 16: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/16.jpg)
MyISAM• fastest read speed• supports FULLTEXT indexes• non transactional• less data reliability
![Page 17: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/17.jpg)
InnoDB• transactional• better data integrity• until MySQL 5.6 it didn’t support FULLTEXT
indexes in TEXT fields
![Page 18: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/18.jpg)
querying into TEXT fields
• inefficient search with LIKE statement• slow DDL operations (like alter table)• unnecessary increase of the table size
![Page 19: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/19.jpg)
possible alternatives
![Page 20: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/20.jpg)
fastest search
![Page 21: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/21.jpg)
search servers• indexes large bodies• api bindings• decoupled
![Page 22: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/22.jpg)
smart retrieval
![Page 23: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/23.jpg)
content delivery network
• high availability• cheap
![Page 24: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/24.jpg)
TEXT field within the RDBMS
![Page 25: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/25.jpg)
![Page 26: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/26.jpg)
conclusion• there is no silver bullet solution• some implementations may add an
additional layer of complexity to the application
• some implementation works better with decoupled applications
![Page 27: Strip your TEXT fields - Exeter Web Feb/2016](https://reader034.vdocuments.mx/reader034/viewer/2022050614/58ac79761a28ab4c6c8b565d/html5/thumbnails/27.jpg)
thanks!come say hi :)