metalayer presentation at techcrunch disrupt ny 2012
DESCRIPTION
The presentation from metaLayer's API workshop at TechCrunch Disrupt 2012.TRANSCRIPT
NY:DISRUPT API HACKATHON
BIG DATA IS
BIG
PUNY HUMANS ARE
PUNY
by Kate Starbird, Bill Morris, Chris Danforth and GeoSprocket
Data Sets + APIs + Visualizations
Made Simple.
Data Sets + APIs + Visualizations
Made Simple.
We’ve got hoodies (as many as you can take)
Prize! Apple TV for best in show using our Image API
Prize! Kindle for best in show using our Text API
We’re hiring!
SCHWAG ALERT
Using the MetaLayer Text API
Our Text API offers methods for extracting contextual features from text documents using various forms of entity extraction.
•Find place names in documents and turn them into lat/lon coordinates for mapping
•Perform auto-tagging and classification using uncommon keywords
•Utilize our sentiment analysis algorithms
Built using pyNLTK (python natural language toolkit)
Brown Corpus (Brown University Standard Corpus of Present-Day American English)
Extended to deal with short form text like Tweets
Corpus can be extended to optimize the API for specific use cases.
60,000 items per minute on a single server
Scales horizontally for ‘bigger data’ streams
ARCHITECTURE
Sentiment Analysis
Tagging (Uncommon Keywords)
Location Disambiguation
Bundle (a Buffet of Awesome)
FUNCTIONS
Response Example: { "status": "success", "method": "bundle", "service": "datalayer", "response": { "datalayer": { "text": "this is some excellent text that needs to be tagged especially if we can pick the location Philadelphia PA out of it", "locations": [ { "latitude": 39.952300000000001, "confidence": 10.0, "name": "Philadelphia, PA, US", "longitude": -75.162400000000005 } ], "sentiment": 0.63960214906683133, "tags": [ "excellent", "text", "location", "Philadelphia", "PA" ] } } }
FORMAT
Post: “This is some excellent text that needs to be tagged especially if we can pick the location Philadelphia, PA out of it.”
FORMAT
Post: “This is some excellent text that needs to be tagged especially if we can pick the location Philadelphia, PA out of it.”
Response Example: { "status": "success", "method": "bundle", "service": "datalayer", "response": { "datalayer": { "text": "this is some excellent text that needs to be tagged especially if we can pick the location Philadelphia PA out of it", "locations": [ { "latitude": 39.952300000000001, "confidence": 10.0, "name": "Philadelphia, PA, US", "longitude": -75.162400000000005 } ], "sentiment": 0.63960214906683133, "tags": [ "excellent", "text", "location", "Philadelphia", "PA" ] } } }
Using the MetaLayerImage API
Our Image API offers methods for extracting contextual features from photos and image documents.
•Find all the objects in a photo along with their relative coordinates in the frame
•Extract color/histogram profiles•Parse text from the image to make it actionable
Built using Teeseract OCR with a proprietary training set.
Extensible and Adaptive
20,000 items per minute on a single server
Scales horizontally
Parses JPG and PNG documents
ARCHITECTURE
Color Profile (all colors in an image)
Histogram (the distribution of color in an image)
Optical Character Recognition (OCR)
Object Recognition
Bundle (Awesome Reloaded!)
FUNCTIONS
MetaLayer Dashboard API*Coming soon young, Padawan.*
Our Dashboard API offers methods for interacting with MetaLayer Dashboard products through scripting.
•Remotely create and configure dashboard environments•Push data to RESTful Dashboard hook (ex. real-time streams) •Poll/Request data from Dashboard hook (ex. historic data)•Analytics from Dashboard activity streams•Create data mashups, consume the JSON output
Dashboard is build with Python and Django
Adds GUI elements to working with APIs
Drag and drop data science. Anyone can use it to mash up data.
Background tasks distributed across horizontal servers.
ARCHITECTURE
Control Administration & User Roles
Modify GUI Features
Data Acquisition
Data Consumption
Analytics
FUNCTIONS
API.METALAYER.COM
