tomer shiri - a bug's life - how to avoid production catastrophes
DESCRIPTION
Lesson learned from production bugs that really happened to Tomer Shiri in his iOS apps. Presented at the TLV iOS Developer Meetup on 16/11/2014TRANSCRIPT
![Page 2: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/2.jpg)
Remove before flight
![Page 3: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/3.jpg)
![Page 4: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/4.jpg)
Who’s to blame?
![Page 5: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/5.jpg)
![Page 6: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/6.jpg)
How to avoid?
Don’t release anything that has local changes
Avoid Pre release checklists completely
Avoid manual configurations
Use build configurations
![Page 7: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/7.jpg)
avoid manual operations
![Page 8: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/8.jpg)
Demo time
![Page 9: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/9.jpg)
Highway 64
![Page 10: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/10.jpg)
![Page 11: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/11.jpg)
![Page 12: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/12.jpg)
What when wrong?
![Page 13: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/13.jpg)
All integers are MAX_INT
![Page 14: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/14.jpg)
How these values age generated?
Configfile
![Page 15: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/15.jpg)
@dynamic properties
![Page 16: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/16.jpg)
Seems reasonable?
![Page 17: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/17.jpg)
Whats ‘c’?
![Page 18: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/18.jpg)
![Page 19: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/19.jpg)
Whats ‘B’?
![Page 20: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/20.jpg)
Solution?
![Page 21: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/21.jpg)
That’s an edge case, man
Have you ever swizzled private methods?
Removed that annoying UIScrollView from Apple’s views?
Used KVM in order to access / change private ivars?
![Page 22: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/22.jpg)
Don’t rely on internal implementation
![Page 23: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/23.jpg)
The overachiever
![Page 24: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/24.jpg)
![Page 25: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/25.jpg)
High traffic!
![Page 26: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/26.jpg)
Some web services stopped functioning.
Oath API getting hammered by the clients
![Page 27: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/27.jpg)
When an API requests fails, it stops all other requests and tries to obtain a new auth_token.
When two threads tries to do that in the same time, they are failing each other.
![Page 28: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/28.jpg)
Solution?
There is no good solution for that.
Threading bug are hard to hunt.
![Page 29: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/29.jpg)
Use kill switches
![Page 30: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/30.jpg)
Prepare for the worst
![Page 31: Tomer Shiri - A Bug's Life - How to Avoid Production Catastrophes](https://reader033.vdocuments.mx/reader033/viewer/2022052910/559c30b01a28abd5468b46b2/html5/thumbnails/31.jpg)
avoid manual operations
Everything that can compile - should compile
Don’t rely on internal implementation
Prepare for the worst