Download - Sharing Code and Experiences
![Page 1: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/1.jpg)
Sharing Code and Experiences
@fabriziomello
![Page 2: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/2.jpg)
About me
![Page 3: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/3.jpg)
I was born in Dom Pedrito in the South of Brazil
![Page 4: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/4.jpg)
But I lived in Bagé since I was 1 year old
![Page 5: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/5.jpg)
Bagé is 236 miles away from Porto Alegre
![Page 6: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/6.jpg)
Porto Alegre is the FISL city
![Page 7: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/7.jpg)
In my hometown Bagé is common people...
● … be born there● … grow up there● … build a family there● … spend the entire life there● … and die there.
![Page 8: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/8.jpg)
Until 2001 this lifestyle fitted with my “old” needs
![Page 9: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/9.jpg)
But I met this pretty woman
and she changed my life
![Page 10: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/10.jpg)
And now I’m a husband and father
![Page 11: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/11.jpg)
● IT experience since 1993○ Programming Languages (Basic, C, Clipper, Pascal,
PHP, Javascript, …)○ Operating Systems (Windows “argh”, Unix and
Linux)○ PostgreSQL, Firebird, MySQL, Oracle○ Agile Methodologies (XP, Lean, Scrum, …)○ …
Background (1/2)
![Page 12: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/12.jpg)
Background (2/2)● Bachelor in Information Systems in 2002
● Entrepeneur at http://timbira.com
● Agile Methodologies Specialization student 2014/2015
● PostgreSQL colaborator since 2008 (Brazilian community and now the international too)
![Page 13: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/13.jpg)
FOSS and me
● My first contact was using Linux in 1997● I fell in love with this culture since then● In 1999 I met PostgreSQL so since then I
knew this would be part of my life● Because of this decision I had a lot of
troubles, including financial…● But here I am :-)
![Page 14: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/14.jpg)
Is a global program that offers students stipends to write code for open source
projects.
We have worked with the open source community to
identify and fund exciting projects for the upcoming
summer.
![Page 15: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/15.jpg)
Connect students to open source communities
![Page 16: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/16.jpg)
GSoC and PostgreSQL
● Since 2006● Cool projects
○ Fast GiST index build○ New phpPgAdmin Plugin Architecture (brazilian)○ pgAdmin database designer○ Better indexing for ranges○ Document collection Foreign-data Wrapper
![Page 17: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/17.jpg)
And now my project ...
PostgreSQL 9.1 introduced a new kind of tableUnlogged Tables
![Page 18: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/18.jpg)
What means “Unlogged”?
First we need to know what means “WAL”
PostgreSQL is Full-ACID and to guarantee data integrity uses a standard method called
WAL (Write-Ahead Logging)
![Page 19: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/19.jpg)
WAL (Write-Ahead Logging)“In computer science, write-ahead logging (WAL) is a family of techniques for providing atomicity and durability (two of the ACID properties) in database systems.
In a system using WAL, all modifications are written to a log before they are applied. Usually both redo and undo information is stored in the log.”
http://en.wikipedia.org/wiki/Write-ahead_logging
![Page 20: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/20.jpg)
Ok, and what means “Unlogged” ?
● Unlogged means that the data written in these tables is not written to WAL.
● So it makes written really, really fast compared to written into regular tables.
![Page 21: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/21.jpg)
So I’ll use it to all of my tables...
● However you won’t want to do that, because
● They are neither crash-safe (an unlogged table is automatically truncated after a crash or unclean shutdown)
● And they are nor replicated using SR
![Page 22: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/22.jpg)
But there are some cool use cases
● Speed ETL jobs● Cache● Session State● Queues?!● ...
![Page 23: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/23.jpg)
And now we have the power to ...
● change from UNLOGGED to LOGGED○ ALTER TABLE name SET LOGGED;
● change from LOGGED to UNLOGGED○ ALTER TABLE name SET UNLOGGED;
![Page 24: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/24.jpg)
Already committed commit: f41872d0c1239d36ab03393c39ec0b70e9ee2a3cauthor: Alvaro Herrera <[email protected]>date: Fri, 22 Aug 2014 14:27:00 -0400Implement ALTER TABLE .. SET LOGGED / UNLOGGED
This enables changing permanent (logged) tables to unlogged andvice-versa.
(Docs for ALTER TABLE / SET TABLESPACE got shuffled in an order thathopefully makes more sense than the original.)
Author: Fabrízio de Royes MelloReviewed by: Christoph Berg, Andres Freund, Thom BrownSome tweaking by Álvaro Herrera
![Page 25: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/25.jpg)
How it works
1. Acquire AcessExclusiveLock2. Check dependencies
a. Cannot change temp tablesb. Check Foreign Keys
3. Change indexes “relpersistence”4. Create new heap/toast with new relpersistence5. Rewrite heap/toast6. Rewrite indexes
![Page 26: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/26.jpg)
New patch with refactoring1. Acquire AcessExclusiveLock2. Check dependencies
a. Cannot change temp tablesb. Check Foreign Keys
3. Create new heap/toast with new relpersistence (pass down relpersistence to reindex_index)
4. Rewrite heap/toast5. Rewrite indexes
![Page 27: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/27.jpg)
Currently Caveats
● AccessExclusiveLock● Rewrite datafiles
![Page 28: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/28.jpg)
Future work
● Don’t rewrite datafiles when wal_level = minimal
● Unlogged Indexes on Regular Tables● Unlogged Materialized Views (was reverted
by Tom Lane because of the bad design)
![Page 29: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/29.jpg)
Questions?
![Page 30: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/30.jpg)
Special thanks to
● Stephen Frost (mentor)● Josh Berkus and Thom Brown (organizers)● Christoph Berg (patch review)● Álvaro Herrera (patch review and commit)● Maristela Kohlrausch de Andrade (my
english teacher)
![Page 31: Sharing Code and Experiences](https://reader033.vdocuments.mx/reader033/viewer/2022052506/557d608ad8b42ae1438b4ff4/html5/thumbnails/31.jpg)