Exploratory Data Analysis & Visualization, Spring 2015:

Intro to APIs and examples in Python

Yu Tian, Christine Lee

March 31, 2015

What we aim to go over today:

Overview to APIs. Why should we care?

How do APIs work?

Indeed.com API quick example with Python/R

Demo of Yelp API with Python/R

- 2 -

- 3 -

What are APIs?

API stands for Application Programming Interface – a way for applications to talk to each other.

APIs are the secret plumbing of the web. They are a way to get the functionality of an application (such as Twitter, Facebook) without visiting the website for that application.

Why is it valuable?

Websites are designed to cater to human's strengths. Yet, what happens when you face a very time-intensive task, like copying the contact info for a thousand customers from one site to another?

You would love to delegate this work to a computer so it can be done quickly and accurately. The solution is an API.

API is the tool that makes a website's data digestible for a computer.

Through it, a computer can view and edit data, just like a person can by loading pages and submitting forms.

- 4 -

Interfaces & servers

APIs create standards for what kind of information applications expect to receive from other applications, and from user interfaces.

One interface might talk to several applications. One application might talk to several interfaces.

For example, twitter.com is just one interface for twitter; facebook.com is just one interface for facebook.

(It helps to remember that the API is simply another program running on the server.)

- 5 -

2 sides in an integration of an API: client & server

When two systems (websites, desktops, smartphones) link up through an API, we say they are "integrated." In an integration, you have two sides, each with a special name.

One side we have already talked about: the server. This is the side that actually provides the API.

The other side is the "client." This is a separate program that knows what data is available through the API and can manipulate it, typically at the request of a user.

A great example is a smartphone app that syncs with a website. When you push the refresh button your app, it talks to a server via an API and fetches the newest info.

- 6 -

An example: smartphone app <-> pizza parlor API

APIs receive “calls” that tell them:

– Who you are

– What you want the API to do

– What it needs to know in order to do that

Example: In this scenario, the pizza parlor API is the server and the smartphone app is the client.

A customer uses the app to choose a pizza and then hits a button to place the order.

As soon as the button is pressed, the app knows it needs to make a request to the pizza parlor API.

- 7 -

Why APIs matter

Before 2010 the concept of API use is not widespread…

Projected Open API Growth Rate (Source: Craig Burton and Phil Windley)

- 8 -

Why APIs matter

Why do companies open up API use?

1) APIs let your product grow faster than you can build it.

Twitter’s case: anyone who wants to build a better product/app with our platform, go ahead -

What happened then: Many developers then built apps/businesses with the Twitter API

What this meant:

1.Twitter users have different ways to access the Twitter system through different apps;

2. The developer ecosystem enabled / help drive growth of the Twitter platform to where it is today.

2) APIs might be important for product launches.

Examples:

- Google+ in its first 2 years;

- Bitly’s bundle product didn’t have an API and hence lost developers’ interest early on.

- 9 -

Limitations with API use

Some key limitations:

Not every platform or company has an API, or useful API.

Examples: Google+ in its first 2 years; Bitly’s bundle product

Some APIs may have limitations or restrictions to full data access

Example: LinkedIn’s API will be restricting full access to select partners

If you are developing an app/business on a platform’s API, you may be at the mercy of that company’s API policy and agendas

- 10 -

APIs made it possible for data to be exchanged in ways that weren’t possible before

OAuth lets an application interact with a platform on behalf of a user.

Application: Favstar.fm

Platform: Twitter

User: You

- 11 -

Oauth example: Syncing AmEx card with Foursquare to unlock savings

AmEx uses Foursquare on your (user’s) behalf.

Which means AmEx can access your Foursquare data, but NOT that Foursquare can access your AmEx data.

Application: AmEx

Platform: Foursquare

User: You

- 12 -

Authentication

We know who the client and server are, and we know they speak in specific data formats to understand each other. Knowing how to talk, though, leaves an important question: how does the server know the client is who it claims to be?

Key terms:

Authentication: process of the client proving its identity to the server

Credentials: secret pieces of info used to prove the client's identity (username, password...)

Basic Auth: scheme that uses an encoded username and password for credentials

API Key Auth: scheme that uses a unique key for credentials

- 13 -

Authentication

There are several techniques APIs use to authenticate a client. These are called authentication schemes. Let's take a look at two of these schemes now.

Basic Authentication - only requires a username and password. Why does using the same username and password to access the API and manage the account is not ideal?

There may be times when the client should have different permissions than the account owner. Take for example a business owner who hires a contractor to write a program that uses an API on their behalf. Trusting the contractor with the account credentials puts the owner at risk because an unscrupulous contractor could change the password, locking the business owner out of their own account.

API Key Authentication - API Key authentication is a technique that overcomes the weakness of using shared credentials by requiring the API to be accessed with a unique key. In this scheme, the key is usually a long series of letters and numbers that is distinct from the account owner's login password. The owner gives the key to the client, very much like a hotel gives a guest a key to a single room.

Demo: Indeed.com API request

A web search on indeed.com Indeed API’s available parameters

- 14 -

APIs receive “calls” that tell them:

- Who you are

- What you want the API to do

- What it needs to know in order

- 15 -

API call in Python – using an example from a Github repo

– create a free Publisher account to receive an API key and access tools to receive job data

– define required parameters

- 16 -

API result output from Python converted to a csv table

- 17 -

Now, the fun part – mapping API data onto geographic map in R

How to visualize Indeed job search results in an interesting way?

Packages used in R: Leaflet

- 18 -

On selecting map tiles / creating custom base maps on Mapbox

Insert text here

– Insert text here

- 19 -

Indeed job search results mapped with Leaflet package in R

- 20 -

Zooming in..

- 21 -

References for this presentation

Sources for some of the case examples an illustrations in the section on Intro to APIs:

– Matt LeMay, “The What and Why of APIs”

– Zapier, Introduction to APIs - https://zapier.com/learn/apis/chapter-1-introduction-to-apis/