email as a datasource
TRANSCRIPT
“The reports of my death were greatly exaggerated”
3.9 billion in 20134.9 billion by 2017183B messages/day
400M tweets/day
Email is a communications system
group collaboration task management document collaboration customer support app notification !
project management client relationship applicant tracking photo sharing bug tracking Nigerian extortion
Overview of protocols and standards
or “which acronym does what”
SMTPProtocol for transmission of emails across the internet
IMAPProtocol to allow a client to access and manipulate emails on a receiving server.
• All messages and their folder organization are on the server
• Clients poll to know about new messages that arrive or actions made through other clients
• While it doesn’t send messages, clients usually store sent messages through it
POPProtocol to allow a client to retrieve emails from a receiving server.
RFC-822 MIME Multipart
Standards defining headers and the actual body of the message
Where does your app fit in there?
Typical things apps want to do with email
1. Send emails to users
2. Receive emails from users
3. Access emails users send and receive.
Email is a communications system
group collaboration task management document collaboration customer support app notification !
project management client relationship applicant tracking photo sharing bug tracking Nigerian extortion
Introduction to IMAP
Me: “App Developer, meet IMAP. IMAP, meet App Developer.” !
IMAP: “I don’t give a shit about you, App Developer. Go away!”
Pitfall #1: Identifying messages
Sequence Number
Unique Identifier (aka UID)
Pitfall #2: No data until you select a mailbox
Pitfall #3: Attachments
Pitfall #4: Deleting messages
The joys of parsing email messages
Yo! I fetched a message! Now what do I do?
A simple message
Delivered-‐To: [email protected] Return-‐Path: <[email protected]> Received: by 10.229.135.136 with SMTP id n8mr410292qct.135.1336583200550; Wed, 09 May 2012 10:06:40 -‐0700 (PDT) Received: from smtp-‐out.amazon.com (smtp-‐out.amazon.com. [72.21.212.39]) by mx.google.com with ESMTP id b2si1383913qcd.195.2012.05.06.40; Wed, 09 May 2012 10:06:40 -‐0700 (PDT) Date: Wed, 9 May 2012 17:06:39 +0000 (UTC) From: Amazon EC2 Notification <no-‐reply-‐[email protected]> To: "Sys Admin" <[email protected]> Cc: "Alerts" <[email protected]> Message-‐ID: <[email protected]‐1.amazon.com> Subject: Notice: Amazon EC2 Instance scheduled for retirement MIME-‐Version: 1.0 Content-‐Type: text/plain; charset=UTF-‐8 Content-‐Transfer-‐Encoding: 7bit !Hello, ...
A message with an attachmentMIME-‐Version: 1.0 Content-‐Type: multipart/mixed; boundary=_MYBOUNDARY_ !-‐-‐_MYBOUNDARY_ Content-‐Type: text/plain !This is the body of the message. -‐-‐_MYBOUNDARY_ Content-‐Type: image/jpeg; name="IMG_713.jpg" Content-‐Disposition: attachment; filename="IMG_713.jpg"; size=6379099; Content-‐Transfer-‐Encoding: base64 !/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAZA+4AJkFkb2JlAGTAAAAAAQMA AwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMD8IAEQgAegG1AwERAI RAQMRAfEASMAAQACAwEBAQEBAAAAAAAAAAAHCAUGCQQDAgEKAQEAAgIDAQAAAAAAAAAAAAAAB gcFCAEDBAIQAAEEAgEBBgQGAQUAAAAAAAUCAwQGAQcAEjBQERMUFRBgFhcgQHAhNAhBMSIjJDURAAIC== -‐-‐_MYBOUNDARY_-‐-‐
A message with alternative parts
MIME-‐Version: 1.0 Content-‐Type: multipart/alternative; boundary=_MYBOUNDARY_ !-‐-‐_MYBOUNDARY_ Content-‐Type: text/plain; charset="us-‐ascii" Content-‐Transfer-‐Encoding: quoted-‐printable !Hello! Here’s a message with *rich* text -‐-‐_MYBOUNDARY_ Content-‐Type: text/html; charset="us-‐ascii" Content-‐Transfer-‐Encoding: quoted-‐printable !<html><body>Hello! Here’s a message with <b>rich</b> text</body></html> -‐-‐_MYBOUNDARY_-‐-‐
Pitfall #1: Message-ID is optional
Pitfall #2: In-Reply-To References
Pitfall #3: Attachments are what you
decide them to be
contacts
filesmessages
threads
webhooks
