don't fear failure
TRANSCRIPT
DON’T FEAR FAILUREA TALK ABOUT SOME OF MY MANY FAILURES
SOME THINGS ABOUT ME
• Most people call me Stabby (@StabbyCutyou)• I’ve never started my own company• I’ve never written any popular or even recognizable software• I am definitely not good at “Computer Science”• I almost dropped a live database once
KEY TAKEAWAYS FOR THIS TALK
• I am a huge failure• Just have a plan• Own it• It’s OK to fail
FAILURE“LACK OF SUCCESS”
Set out to do a thing
Did it work?YES NO
FAILURE LEVEL:
EXPERT
OVER 12 YEARS OF MY LIFE
• .NET, Javascript, Ruby, SQL, Go• Love what I do• Screw up tons of stuff in all the time• Manage to get enough right to get by
LESSONS TO LEARN
FROM FAILING
LESSON ONE:
ITS OK IF YOU’RE NOT A SUCCESS STORY
OUR CAREERS ARE DEFINED BY “SUCCESS”
• Large valuations• Perks• Bonuses• Exits• Renown
Most of won’t have that kind of success
I THOUGHT MY FIRST JOB WAS GOING TO BE “IT”
• Guess how that went• One day got an email my company sponsored gym
membership was cancelled• Next day, company was closing down• Had to tell a recent hire who just started two weeks ago that
no, this was not a joke.
I LEARNED A LOT ON THE WAY DOWN
• Built a solid set of SQL skills• Learned a lot of ways you probably shouldn’t build a desktop
app• Exposed to running a dreadful meeting every week• First encounter with bizarre interoffice politics / relationships
LESSON TWO:
NO MATTER HOW HARD YOU WORKYOU CAN’T ALWAYS STOP FAILURE
FOR EXAMPLE… AWS
• AWS is great• Network can be unreliable• Resiliency / Durability in your stack only goes so far• Partitions happen
ANOTHER JOB
• Built the worlds worst Foodler clone• Amongst the worst times in my life• Barely slept, constant rewrites, no guidance or design• It had no chance in the market
There was no way to save it
It was simply destined to fail
I beat myself up over it for a long time
LESSON THREE:
ASK FOR HELP
IT’S OK TO NOT KNOW WHAT YOU’RE DOING
• No one really does at first• Many of us still don’t• Find someone who does
FOR EXAMPLE… RIAK
• Built around the idea of replication• Writes get copied to 3 nodes (by default)• Highly Available - data always accessible• Conflict resolution: Siblings, Vector Clocks
WE WERE CONVERTING TO A NEW JOB SYSTEM
• It kept hanging• I would wake up every few hours to check it• Worked with other engineers to solve the issue• Figured out how to predict it, put in mitigations• Realized no-one cared so long as it got fixed
LESSON FOUR:
OWN YOUR MISTAKES
FOR EXAMPLE… GO
• Writing a lot of Go• Lots of gripes• Fell in love with error handling in Go• Responsibility strictly delegated upward until you “care”
WE TRIED TO BUILD OUR OWN QUEUE
• Sure was fun• But did not succeed like we hoped• Learned a lot of great information• But ultimately wasted a lot of time and effort
LESSON FIVE (a):
IT’S OK TO FAIL…
LESSON FIVE (b):
…SO LONG AS YOU HAVE A PLAN
FOR EXAMPLE… TCP
• Built around the idea of missing data• User land VS Using your brain• Defends against problems in-depth
I MOVED A BUNCH OF FINANCIAL DATA
• This was a big one• We moved some finance data into a more scalable system• During the migration some data was initially lost• We didn’t catch it• We paid out accounts based on incorrect data
WELL
I’M DEFINITELY FIRED
BACK TO MY MAJOR SCREWUP…
• Multiple copies of the missing data• We built scripts to compare datasets and find missing records• We repopulated the missing data• We worked with finance to fix the numbers• Did not get fired
CULMINATION OF ALL LESSONS LEARNED
• We didn’t succeed like we hoped• We had multiple layers to verify, but not enough• We had a plan incase this happened• Asked for help from the right folks• Stepped up, personally accepted responsibility immediately
Set out to do a thing
Did it work?YES NO Did you learn?
YES
NOTry Again
IN THREE DAYS
I START A NEW JOB
LESSONS
• Not everyone is a success story• Failure is inevitable• Ask for help• Own your failures• It’s ok to fail, just have a plan for it
THANKS
@StabbyCutyou